Tag Archives: exam results

Piloting a surgical approach to checklists

cockpit-79525_640On boarding a plane I invariably find I’m looking in through the side window of the cockpit where one of the pilots is likely to be holding a pad and systematically going through the pre-flight checklist. I’ll wonder, ‘Don’t they get bored with going through the same set of checks day after day?’……and then…. ‘I do hope they’ve gone through the check properly today. Especially today.’

I was struck by an article I read in the paper a few years ago about how operating theatres were seeing dramatic reductions in patient post-operative issues by the simple use of checklists. Promoted as an essential strategy by Prof. Stephen Bolsin  (the whistle-blower consultant anaesthetist at Bristol Hospital who re-located to Australia after he lifted the lid on the higher-than-tolerable mortality rates on the cardiology wards) the use of checklists was resulting in a dramatic fall in death rates. The procedure has been so dramatic that in the last few years the World Health Organisation has taken up the use of pre- and post-operative checklists and survival rates are significantly higher than before their use.

checklists save lives

So, teaching is not as immediately life-or-death as flying Jumbos or opening up patients. But we are engaged in a procedure subject to repetition. The annual cycle of preparing, delivering, practising and assessing can become familiar, then automatic, then – unconscious. And when it’s the latter things can fall off the table in amongst all the initiatives, directives and additions. That’s when we’re liable to drop the ball, forget to cover an essential aspect or – worse – consider it, realise we don’t have time for it, and dismiss it as not really important anyway.

That’s why I think checklists are valuable. Yes, they can feel like the bureaucrats are curtailing the impulsively creative instinct but sometimes good management is about sustaining best practice as much as initiating the novel. Maintaining can feel mundane – except that for each cohort we take through the process, it’s their first (and only) time. So it’s incumbent upon us to distil the essentials, package them coherently, and make sure each fresh set of faces gets access to the best we can offer. Time, and time, and time again. It’s a case of building up experiential capital.

A few years ago we experienced some not very fulfilling exam results across the school. Talking through the issues it became apparent that some of those procedures you think would be bread and butter were not being enacted school-wide. There were cases of students not knowing when their exam was until a few days before (how can you plan a revision programme?); some had relatively little material to revise from; others hadn’t been exposed to many practice questions with the marking scheme in front of them. Those things you assumed were happening in every department, clearly weren’t. In working on some ‘good practice’ CPD I remembered the article about surgical checklists and, in consultation with heads of subject, developed a departmental checklist to help give some procedural advice. ‘Quality assurance’ is a cliched term – but when it’s absent the injustice doesn’t affect the deliverers so much as the recipients. You will find the subject-leader’s exam preparation checklist we constructed here.

Subject leader checklist

Though no-one is fail-safe. This year it was my deparment whose results bombed. It goes something like this:

You sit waiting looking at the screen. The third cup of tea has lost its appeal and is barely tepid. You awoke at 4.40 a.m. , tried to go back to sleep,  gave up and have been filling time since then.  You try the site again, just in case someone has blundered (or taken pity) and put them up before the 6.00 a.m. official start. The sun is skitting the roofs of the houses opposite, later than it has been in this fourth week of August but still promising a good day. It’s a curious mix of (blatant) anxiety and (withheld) anticipation that eases not a jot with each year you have done this. You don’t get used to exam results day; each one carries its own melee of, hopefully, personal victories; mini-battles with the fates that you achingly want to see imprinted on those cold slips of paper. You can’t help yourself envisaging the faces of those students whose results will be amongst the first you look for to slay the dragon and win the favour.

Then they’re up – and they’re woefully disappointing. Yes – you’ve caught sight of an A there  (shame it wasn’t an A*) …. a flurry of B’s – yes…. a C for her, great – she so deserved that….. but they’re thinly spread. The Ds and Es blend into a muted background of ‘stuff’. And, strewth, there’s a sprinkling of Fs and Gs in there too. Within moments all the hoped-for exhilaration slides off you, evacuating as quickly as bravery from a lost battlefield, and you’re left with…… feeling gutted.

Do this gig enough times and there will be a year when your department’s exam result pan. Not just poor – but unexpectedly, unpredictably so and like a flattened hand-swipe into your stomach. You’re still recovering as you drive into school, rehearsing the responses you’ll fan back to the eager queries of successful colleagues – ‘Oh – and how were yours?’  Yes- very much as we expected….. Some good individual results in there we’re really pleased with….. Oh, you know, in line with our predictions. Liar.

It has to be gone through and the eviseration has to claim some pain. We really thought that they would hit the heights. Three groups of fabulous students, three experienced teachers at the top of their game and going into the exam with 70% at A*-C with 50% of the marks in the bag. The kids were more on-board than ever, delighting us with their competing cries of how many revision flash-cards they had made and launching themselves down our corridor immediately after the two exams to crow how it had been ‘easy’, ‘fine’, ‘answered all the questions Sir – with time to check them over’. This was the year we would get our best ever and surpass national average with clear blue water. In the event we hit below nat. av. A*-C from 73 students. There were some fine individual performances, real montages of joy and celebration; but too few, too scarce.

So this is what we have done. This is the post-mortem but turned into a exercise in deep introspection, self-reflection, data analysis and – eventually – building something better out of the pathology. The results were analysed to the nth degree. By gender, by class, by sub-group, by gender sub-group…. The question-by-question profile comparing the answers of our students against similar centres and national average that AQA offers was ripped open and the entrails examined. Results by question, by paper, by tier were compared with those from the last 3 years (Message: no discernable pattern). We got papers returned and papers remarked (with a majority upgraded and one raised by a full 10%). But, the action we engaged in that I sense will have most lasting impact was the self-reflection checklist we all filled in. It was deliberately provocative, posing hypotheses that we hadn’t done our job properly. There needs to be a lot of mutual trust to do this one – but the answers that came out from it were deeply reflective and practice-changing. To 95% of the ‘hypotheses’ we all checked them off to say – no, did that. But it’s the narrowing down to the 5% that will mean this year’s cohort are better prepared. All three of us came up with something different that, on reflection, we will change. It’s a checklist to find that needle in the bloody haystack.

And the checklist I’m working on for later this year is for the induction of new subject-leaders. I’m concerned we make a new appointment and, effectively, throw them in the deep-end. And then get disappointed in seeing them struggle. An effective induction programme should be the result of widespread discussion, a clarification of the key skills the role will draw upon, an assessment of the level of competence required in those elements and a matching of an experienced ‘buddy’ or mentor to ensure the plane lands with its undercarriage down and the wheels spinning. This is what has been developed for meeting likely needs in the first term after a September appointment.

Induction checklist

Checklists don’t sound sexy. But purposeful reiteration is ensuring the best we offer doesn’t dim with overfamiliarity. And if it means planes don’t crash and patients don’t relapse, maybe they can do something for the quality of experience students and teachers can anticipate. To adapt Bill Shankly’s infamous phrase: “Teaching – a matter of life or death? No – it’s more important than that, son”.

This was the basis of a short presentation I gave at #SLTcamp. As an initiative in personal career development it surpassed all expectations. The professionalism and care that @mrlockyer  and @MsFindlater extended in the organising, advertising, managing and hosting of this event was testimony to all that is best in this profession. It should anchor its worth by becoming an enduring legacy that provokes similar extended sleep-over TeachMeets in other parts of the country in the coming months and years. I sincerely hope so.

I shall be checking.


Filed under School management, Uncategorized