Categories
Pivot

Getting the Dumb Stuff Right

Checklists may not be what you think they are. They are a powerful tool for eliminating errors and have been shown to be effective in improving the safety in aeroplanes, surgery and building skyscrapers. But effective checklists are not simply a list of all items to be done or checked…

On October 30, 1935 at Wright Air Field in Dayton Ohio, the US Army Air Corps held a flight competition for airplane manufacturers vying to build the military’s next-generation long-range bomber. It wasn’t supposed to be much of a competition. In early evaluations, the Boeing Corporation’s gleaming aluminium alloy Model 229 had trounced the designs of Martin and Douglas. Boeing’s plane could carry five times as many bombs as the army had requested; it could fly faster than previous bombers and almost twice as far. The flight “competition” was regarded as a mere formality.

A small crowd of army brass and manufacturing executives watched as the Model 229 test plane taxied onto the runway. It was sleek and impressive, with a 103-foot wingspan and four engines jutting out from the wings, rather than the usual two. The plane roared down the tarmac, lifted off smoothly, and climbed sharply to three hundred feet. Then it stalled, turned on one wing, and crashed in a fiery explosion. Two of the five crew members died, including the pilot, Major Ployer P. Hill.

An investigation revealed that nothing mechanical had gone wrong. The crash had been due to “pilot error”, the report said. Substantially more complex than previous aircraft, the new plane required the pilot to attend to four engines, each with its own oil-fuel mix, the retractable landing gear, the wing flaps, electric trim tabs that needed adjustment to maintain stability at different airspeeds, and constant-speed propellers whose pitch had to be regulated with hydraulic controls, among other features. While doing all this, Hill had forgotten to release a new locking mechanism on the elevator and rudder controls. The Boeing model was deemed, as a newspaper put it, “too much airplane for one man to fly”. The army air corps declared Douglas’s smaller design the winner. Boeing nearly went bankrupt.

Still the army purchased a few aircraft from Boeing as test planes, and some insiders remained convinced that the aircraft was flyable. So a group of test pilots got together and considered what to do.

What they decided not to do was almost as interesting as what they actually did. They did not require Model 299 pilots to undergo longer training. It was hard to imagine anyone having more experience and expertise than Major Hill, who had been the air corps’ chief of flight testing. Instead they came up with an ingeniously simple approach: they created the pilot checklist.

...

The test pilots made their list simple, brief and to the point – short enough to fit on an index card, with step-by-step checks for takeoff, flight, landing and taxiing. It had the kind of stuff all pilots know how to do. They check that the brakes are released, that the instruments are set, that the door and windows are closed, that the elevator controls are unlocked – dumb stuff. You wouldn’t think it would make that much difference. But with the checklist in hand, the [test] pilots went on to fly the Model 229 a total of 1.8 million miles without one accident. The army ultimately ordered almost 13,000 of the aircraft, which it dubbed the B-17.

From The Checklist Manifesto by Dr Atul Gawande.

Checklists went on to become a staple for pilots everywhere.

Checklists: They May Not Be What You Think

Checklists are a powerful tool for eliminating errors, and not just in aeroplanes. They have been shown to be effective from surgery to skyscrapers.

In his book The Checklist Manifesto, Gawande quotes measurements collected from surgeries performed around the world before and after a checklist was introduced. The results were stunning.

  • Major complications were down by 36%
  • Infections were down by approximately 50%
  • The number of patients returned to surgery because of problems declined by 25%
  • 150 fewer patients than normal suffered harm from surgery (measured over 4,000 patients)
  • 27 fewer deaths (47% drop) caused from surgical complications

All because of a simple checklist.

But to put them to work effectively, we first need to be able to distinguish between the types of problems we can be faced with, and the types of mistakes we all make.

Problems

Researchers have proposed that there are three different kinds of problems: simple, complicated and complex.

  • A simple problem is one where we may have a few basic techniques to learn, but if we follow the steps we have a high likelihood of success, like enrolling a child. Once we know what to do, we can repeat it. Every time.
  • A complicated problem is one that can sometimes be broken down into a series of simple problems, but there is no straightforward process. Success usually requires multiple people and specialised expertise. Unanticipated difficulties arise and timing and coordination are important. But once we learn, we can repeat the process and perfect it, like organising a centre excursion or a parent evening (or preparing for a centre audit!).
  • A complex problem is one where previous experience and knowledge do not guarantee future success. The outcome is uncertain from one event to the next, like settling an upset child. We know it’s possible to do, but it’s complex!

In our opening example, the pilot was resolving what for him was a simple problem – checking the plane before taking off. He just needed to perform a series of steps to achieve success and he was well trained and experienced in performing them.

He made a ‘dumb’ mistake.

Mistakes

Mistakes are caused by either:

  • ignorance, where we just don’t know enough to do better, or
  • ineptitude, where we do have the knowledge but we just don’t apply it.

Our pilot’s ‘dumb’ mistake was an error of ineptitude. He knew what to do, he just didn’t remember to do it.

It was a simple problem, and an error of ineptitude.

That's where we put our checklist to work!

But Make it a Checklist (Not a Recipe)

We'll define a recipe as a complete list of every step involved in a process or task. A checklist is a listing of the most important steps.

If we gave our pilot a list of every step they needed to perform before take-off, the ‘recipe’ would be long. It would also contain many trivial as well as critical steps, and human nature being what it is, the list would soon fall into disuse (if we were not there to watch). Our pilot would resent the time wasted on all the pointless stuff.

In effect, our 'recipe' would be trying to overcome an error of ignorance where ignorance didn’t exist. (It’s not hard to guess how successful a take-off ‘recipe’ would be in the hands of someone who didn’t have any knowledge in the first place!)

There is a place for recipes, but if we want to have a simple task to be repeated, by someone who already has the knowledge, and without our continuous oversight, we need a checklist - not a comprehensive list of every step, but a list of the most important steps.

Checklists are about ineptitude, recipes are about ignorance.

Creating Our Checklist

In his book, Dr. Gawande highlights the following practices for successful checklists.

Step 1: Limit the Length.

Checklists should contain between 5 and 9 items only, and focus on the most serious or damaging steps to skip. We can leave out anything that is likely to be triggered by another item in our checklist or any items that are so automatic they are never missed. We are looking for the key steps that we don’t want overlooked, or that make us look unprofessional if skipped.

The checklist should be able to be completed in 90 seconds or less; beyond that, it will become a distraction and people will start shortcutting.

If our checklist pushes beyond those boundaries, we should break it down into smaller checklists.

Step 2: Use Appropriate Language.

The wording in a checklist is important. It must be:

  • Simple
  • Clear
  • Exact

We should use the familiar language of the people using the checklist.

Step 3: Define a Clear Pause Point

If we don’t define a clear trigger for using a checklist, it won’t get used.

The trigger is called the pause point. For pilots, a preflight checklist is used before flight. The pilots pause before taking off and conduct their checklist.

And it’s the pause point that distinguishes between the two kinds of checklist:

  • A READ–DO checklist is used where a series of tasks are completed as they are checked off. The pause point is at the beginning.
  • A DO–CONFIRM checklist is used when a series of tasks is done from memory and then checked. The pause point is at the end.

The pause point is often overlooked, yet it is crucial to the successful design and use of a checklist.

Step 4: Think about the Layout.

If a checklist is cluttered or difficult to read it is likely to be misunderstood or put aside as too complicated. The look really matters.

Here are five design standards found to apply to successful checklists:

  • Fit the checklist on one page
  • Keep it free from clutter
  • Use no unnecessary colours
  • Use both UPPER and lower case
  • Choose a sans serif font, like Arial

Step 5: Test and Revise.

Carefully designed checklists nearly always fall apart in the real world!

To make sure our checklist is relevant, effective and up to date, we must test, revise and test again until it fits and works.

Where to Now?

There are numerous areas where a checklist could be used in our centre:

  • Enrolling a child
  • Inducting a new staff member
  • Planning centre excursions
  • Conducting parent centre tours
  • Child's last day
  • Emergency evacuations

We start by thinking about the pause point (when we want the checklist to be used). That determines the type of checklist we need: a READ-DO or a DO-CONFIRM. From there, it’s a matter of itemising the critical steps and following the guidelines above.
[spacer height="5px"]


[spacer height="5px"]

Pre-First-Day Phone Call with Caregiver

Pause Point: Phone call one work day before start date.
Type: READ-DO
Critical Steps:

  1. Confirm child hours
  2. Parking
  3. Etiquette (i.e. no shoes)
  4. Signing-in procedure
  5. Where to leave the child's bag and food
  6. Who to ask for on arrival
  7. Location of adult toilets
  8. Settling/departure expectations
  9. Payment expectations

[spacer height="5px"]


[spacer height="5px"]

Food for Thought...

  • Checklists help us ensure simple tasks are guarded against errors of ineptitude
  • Checklists are not a comprehensive list of every step, just the most important ones
  • Checklists should fit on one page and contain 5 to 9 items
  • Checklists must have a clear pause point or they won't get used
  • Checklists are either READ-DO or DO-CONFIRM
  • We must test and tweak our checklists

Books

2 replies on “Getting the Dumb Stuff Right”

Leave a Reply

Your email address will not be published.