Darren Smith, Flight Instructor
  Home | Login | Schedule | Pilot Store | 7-Day IFR | IFR Adventure | Trip Reports | Blog | Fun | Reviews | Weather | Articles | Links | Helicopter | Download | Bio

Site Map

Subscribe

Private Pilot
  Learn to Fly

Instrument Pilot
  7 day IFR Rating
  IFR Adventure

Commercial Pilot

Multi-Engine Pilot

Human Factors/CRM

Recurrent Training

Ground Schools

Articles

Privacy Policy
About Me

Keyword:
  

Support this Website

Criticism of Memory Item Checklists

by Darren Smith, CFII/MEI
General Aviation Human Factors, February, 2009
Navigation:  Fundamentals of CRM | Resolving Conflict | Workload Management Checklist Usage | Briefings & Callouts | Training CRM | Threats to Safety | Intro to TEM | Error Management | Integrating Threat & Error Management | TEM Countermeasures | What are you doing over there? | New Captain Series

 Checklists:  Intro to Memory Item Checklists | Criticism of Memory Item Checklists

In the previous article, we discussed Memory Item (Recall) Checklists and their uses in airline flying.  In this article, I want to share criticism of the use of Memory Item (Recall) Checklists. 

1. Memory Item Checklists rely on the identification of a specific problem and provide a specific answer to that specific answer.  In an emergency situation, Memory Item Checklists provide the pilot a quick answer to a problem that looks like what was experienced in training.  Occasionally this leads to solving the wrong problem.  This is thought to have occurred on occurred on 8 January 1989, when British Midland Flight 92, a Boeing 737-400, crashed onto the embankment of the M1 motorway near Kegworth, Leicestershire, England, United Kingdom. The aircraft was attempting to conduct an emergency landing at East Midlands Airport. 47 people died and 74 people, including seven members of the flight crew, sustained serious injuries after the flight crew used a Memory Item Checklist to shut down the wrong engine.  The crew had failed to process the checklist cognitively and ignored vibration gauges which were assumed to be inaccurate. As a result of changes between the -300 and -400 model and lack of operating experience, over reliance on Memory Item Checklists is thought to have led this crew into solving the wrong problem.

It is important to note this flight crew was trained with the Boeing syllabus. The 12 day syllabus consisted of training in normal and abnormal procedures in a full flight simulator.  Boeing training for abnormal conditions is that pilots would evaluate all information and take action after the problem had been positively identified.  On the incident flight, the crew had executed an engine shutdown checklist and followed-up with the British Midlands QRH 1.17.2.3.  By the time they got to the QRH, the error had already been made and the steps in the QRH sought to convince them further the correctness of their actions:
“If the emergency is positively corrected, the Captain should evaluate the situation before proceeding with the next step.  If any doubt exists as to the condition of the engine or fire warning system, complete all recall items.”  Source AAIB, UK http://www.aaib.gov.uk/cms_resources/4-1990%20G-OBME.pdf

If the crew did not get the correct result the first time, they would have completed the recall items (Memory Item Checklists) again hoping to get the correct result, rather than “cognitively processing” a checklist.

This isn’t the first time that flight crews have repeated Memory Item Checklists again and again without result.  BA009 (volcanic ash encounter) executed the Engine Restart Checklist about a dozen times without success and it wasn’t until they got to clear air that they were able to get some of the engines restarted.

Pinnacle 3701 executed the 4 item Dual Flame Out Memory Item Checklist all the way from 410 to the crash site hoping to get the engines restarted.  The NTSB stated, “Simply adhering to standard operating procedures and correctly implementing emergency procedures would have gone a long way to averting this tragic accident.”  Had the flight crew “cognitively processed” the emergency instead of relying on the Memory Item Checklist for the problem they were dealing with, they would have recognized the importance of maintaining a minimum airspeed to keep the engine cores rotating to prevent core lock.  This crew gave up basic navigation & airmanship to execute this Memory Item Checklist.  After recovering from the stall at 410 and unusual attitudes passing through 370, the flight crew emphasized bringing engines back online rather than navigating to a safe landing.  Memory Item Checklists are so fundamental and basic to line pilot training that over-reliance on Memory Item Checklists could very well set pilots up for failure.  This crew attempted to make an emergency landing at the Jefferson City, Missouri airport but crashed in a residential area about three miles south of the airport.

2.  Checklist selection errors are common and lead pilots into making missteps rather than correctly diagnosing the problem.  This would be less likely if cognitively processing checklists.

Swissair 111 when confronted with smoke in the cockpit immediately executed Air Conditioning Smoke, a checklist selection error.  What we know today is that the fire was caused by an electrical fire just aft of the cockpit and was raging through the plenum.  By turning on the exhaust fans, the pilots were feeding the flames and ultimately decided to land only when parts of the overhead were melting on the flight crew.

Air Transat 236, an Airbus 330 crew, was confronted with low oil temperature and high oil pressure on engine no. 2 and assumed them to be computer malfunctions and communicated with dispatch.  Twenty minutes later, the pilots were notified of a fuel imbalance and executed the memory item for the imbalance.  What the pilots did not understand at that time is they had been leaking fuel for the past hour.  Because the fuel leak in the starboard engine had still not been diagnosed, this diversion had the effect of sending fuel to the leak and causing further loss.

When presented with an abnormal situation, will often forego a thorough analysis of the situation because pilots will typically focus on the prominent cue. A study conducted by an Embry Riddle student at Delta Airlines Training Center in Atlanta found that a third of line pilots who were presented with an engine start condition with no oil pressure indications selected the wrong checklist.  This natural tendency to fixate on the most prominent cue is a common thread through incident scenarios where crews used Memory Item Checklists to no avail.  This is clearly an area which further research could be conducted.

3. In the human factors field, we know that memory is faulty.  Whether it be long term memory or short term memory, there are certain conditions which can impair, block, distort, or disturb the recall of tasks from either short term or long term memory. 

Just a few studies include:
A.  In the June 1999 issue of Archives of General Psychiatry, researchers at Washington University School of Medicine in St. Louis “provide the first direct evidence that several days of exposure to cortisol at levels associated with major physical or psychological stresses can have a significant negative effect on memory.”  Cortisol is produced by the body during stressful times.  For example during mergers, seniority list integrations, and economic conditions.  It only takes a few days of severe stress or psychological trauma in order for cortisol to produce memory impairment.  The good news: “We don't believe the memory impairments we saw in this study are in any way associated with an irreversible process. In fact, our evidence shows that this memory impairment is quickly reversible.” 

B. From Behavioral Neuroscience, Volume 122, Issue 3, p.697-703 (2008), researchers Tollenaar and Elzinga conducted delayed cue research and recall under stress.  The scenario conducted by these researchers had participants memorize and commit to long term memory a series of word pairs.  The group that had retrieved words under stress 5 weeks after encoding performed worse on long-term recall than the comparable control group that was not under stress.  This situation is compared to learning a memory item checklist during a continuing qualification cycle and needing to use that memory item checklist in an emergency 5 weeks later.

4. Memory Item Checklists exclude the possibility of thoughtful analysis before committing to an action that could make the problem worse or by attempting to solve the wrong problem. 

On November 10, 2008, Ryan Air flight 4102 from FRA to CIA (Rome Ciampino), the FO was Pilot Flying and had experienced a single engine failure due to a bird strike during an approach.  He immediately began the Memory Item Checklist which was go around (throttles forward, nose up), but not yet retracted the flaps when the second engine quit. The Captain took the controls and forced the now engineless aircraft onto the runway.  A few seconds delay would have meant the 737-800 would have crashed into the residential area behind the runway.  The elapsed time between the first engine failure and full stop on the runway was 25 seconds.  There was clearly no time for a checklist, cabin preparation for an emergency landing or communication with ATC.

During the training at an airline that utilizes memory item checklists, the triggers given lead flight crews to a specific memory item checklist to solve a specific failure.  A specific stimulus leads to a specific result and excludes analysis and forethought before committing to a course of action.  If during that training session, the flight crew would “cognitively process” a checklist, the flight crew diagnoses and considers each step as the tasks are performed rather than blindly following a rote memory item checklist.  The emphasis is directed at the process of handling the emergency rather than executing a checklist.  It’s an important cultural difference. 

In this particular scenario, the Captain’s experience as a glider pilot kicked in rather than a Memory Item Checklist.  In this case, it was a right decision. 

Another problem with immediate action items or Memory Item Checklists is that it may not account for incompatible circumstances.  As an example, the B767-300ER Memory Item Checklist states in the event of an 'Engine Overheat' turn off the engine bleed air on the effected side. The Memory Item Checklist however does not take into consideration, nor warn the pilot that doing so above 350 might lead to cabin decompression. 

5.  Memory Item Checklists support tunneling of attention. 

Researcher Baddeley, in British Journal of Psychology, presented a review of studies that included performance of deep-sea divers, combat aviators in actual combat, soldiers in simulated emergencies, and skydivers. These studies evaluated the performance of manual dexterity tasks, tracking tasks, and attention to peripheral cues. They showed that danger manifests itself in human performance through a narrowing of attention or through an increase in time to complete a manual dexterity task. The narrowing of attention can potentially lead to increased performance only if the task being performed is understood to be important.  The only scenario which leads to increased performance is through cognitively processing the checklist used to complete the task.

Researchers Salas, Driskell, and Hughes in Stress and Human Performance, state that if the task is so complex as to require attention to numerous cues, the narrowing of attention will result in an inability to integrate relevant task information and an inability to conduct a proper assessment of the situation.  Such narrowing of attention is the result of relying on Memory Item Checklists while expanding attention is the result of “cognitively processing” a checklist.  These researchers go on to state, under stress, subjects are less effective and more disorganized at considering alternative solutions and incorporate less data in decision-making.  This would support the notion of forcing a pilot to cognitively process a checklist rather than executing a Memory Item Checklist which would reinforce the pattern of incorporating less data in decision making. 

The antidote for such a situation is using the checklist to draw crew attention to environmental or situational cues that support or contradict a diagnosis assuming cues are specified that flight crews are able to assess.  Memory Item Checklists do not support the divergent thinking required to properly diagnose, select the proper checklist, and execute the steps.

In closing, a meaningful context produces a robust training curriculum for pilots and inoculates pilots against memory impairment under stress.  Memory item checklists are stored in procedural memory (in the cerebellum).  An important feature of Scenario based training is to provide a meaningful context in which the emergency occurred.  By focusing on the process rather than the result, more areas of the brain are involved in encoding the training session to long term memory.  Such training sessions which focus on the process rather than a defined result involve more areas of the brain such as the emotional memory in the amygdala, the visual memory in the occipital lobe, and the auditory memory in the temporal lobe.  As such, the neural connections created by such training sessions are stronger and the learning effect more permanent even under stress.

 "The pilot who teaches himself has a fool for a student." — Robert Livingston, Flying the Aeronca 

Your Thoughts...

Name: (Anonymous posts deleted)

E-mail: (if you want a reply)

How did you hear
of this website?
Message:  (What should I write?)
Business Card
WebSearch
News Group
Safety Seminar
Advertisement
Word of Mouth
Other(specify)
(Required) Enter number from image to send:

    

Check this out...
  Home | Login | Schedule | Pilot Store | 7-Day IFR | IFR Adventure | Trip Reports | Blog | Fun | Reviews | Weather | Articles | Links | Helicopter | Download | Bio
All content is Copyright 2002-2010 by Darren Smith. All rights reserved. Subject to change without notice. This website is not a substitute for competent flight instruction. There are no representations or warranties of any kind made pertaining to this service/information and any warranty, express or implied, is excluded and disclaimed including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose. Under no circumstances or theories of liability, including without limitation the negligence of any party, contract, warranty or strict liability in tort, shall the website creator/author or any of its affiliated or related organizations be liable for any direct, indirect, incidental, special, consequential or punitive damages as a result of the use of, or the inability to use, any information provided through this service even if advised of the possibility of such damages. For more information about this website, including the privacy policy, see about this website.