What if Feedback wasn’t all it was cracked up to be.

Boxer 2

So, what if Feedback isn’t all it’s cracked up to be? No, that’s wrong. Instinctively as well as experientially we know the value of feedback. But it must be more subtle than it seems to come across at times when ‘More is Better’ has become an imperative leading to a feverish enthusiasm for work & feedback scrutinies and the associated rabid marking schedule. Maybe a question for its twin – marking.

What if Marking doesn’t satisfy the expectations made of it? Or has a neutral effect? Or what if it has a negative effect, even? All those hours (and extra hours) under which teachers are labouring into late-night and cross-weekend marking sojourns. What if the effect produced doesn’t justify the lugging of books from school to car, and car to study as frequently as they are?

Why the recent (in the last two years, anyway) upscaling of regular marking to the ark of sanctity? Who says Feedback is the great purveyor of progress ? Thank you, Education Endowment Foundation, John Hattie and others. The table says it all, and all that needs to be said is stark: Feedback has the biggest single impact on student progress for minimum cost. Pupil cost. The bit about Teacher cost doesn’t appear to be factored in. Never mind – if we’re dedicated and want the best for our students, then like Boxer, the cart-horse in Animal Farm, we will rise earlier and rest later with the imperative: “I will Work Harder”.

EEF Feedback

Except that the EEF has some further things to say about Feedback and it’s not that ‘more is better’.

Feedback studies tend to show very high effects on learning. However, it also has a very high range of effects and some studies show that feedback can have negative effects and make things worse. It is therefore important to understand the potential benefits and the possible limitations of this as an approach.  The research evidence about feedback was part of the rationale for Assessment for Learning (AfL). One evaluation of AfL indicated an impact of half of a GCSE grade per student per subject is achievable, which would be in line with the wider evidence about feedback. Other studies reporting lower impact indicate that it is challenging to make feedback work in the classroom.

And the strength of the evidence?

There are a substantial number of reviews and meta-analyses of the effects of feedback. Educational (rather than psychological or theoretical) studies tend to identify positive benefits where the aim is to improve learning outcomes in reading or mathematics or in recall of information. The most recent meta-analysis of studies focusing on assessment for learning in schools indicates the gains are more modest, suggesting an improvement of about three months additional progress is achievable in schools or nearer four months when the approach is supported with professional development.

‘Three months additional progress’ places the impact in amongst ‘summer learning’ and ‘outdoor adventure learning’. Not quite the audacity of promise there then.

What is the source of evidence of Feedback having such a positive impact? Guided by John Hattie and the whole of chapter 7 of ‘Visible Learning for Teachers’ which is devoted to the impact on progress that effective feedback can have, Hattie identifies an effect size of 0.79 for feedback. He states : “This places feedback in the top ten influences on achievement, although there is considerable variability.”

Searching for evidence of what makes ‘effective’ rather than ‘not very effective’ feedback, step forward – John Hattie, along with Helen Timperley and their 2007 paper ‘The Power of Feedback’  on which much of chapter seven is based. In their review of academic studies of feedback they start from the presumption that the purpose of feedback should be to guide the student from where they are now to close the gap towards the intended learning goal. What does that best? Some of these were a surpriseHattie Table 2

The conditions that make feedback more or lesson likely to have benefit are as follows:

Feedback is more effective

  • when it provides information on correct, rather than incorrect responses, and when it builds upon a change the student has made.
  • when learning goals are specific and challenging enough to entice a student’s commitment, but the task complexity is low.
  • when learning goals are clearly defined for the student, and the student accepts them as being valid (i.e. worth the investment of effort).
  • if it features as written comments rather than grades. (Does using ‘marking codes’ constitute a ‘grade’ in the students’ eye – or do they transform it into a comment?)
  • when a student is confident they have the correct response, and it turns out to be incorrect.
  • when it identifies the ‘what’ and the ‘how’ students demonstrate comprehension and clarifies the strategies they must take to improve.
  • when it is clear, purposeful, meaningful, compatible with the student’s prior knowledge and provides logical connections towards ways to improve.
  • when the feedback offers little threat to a student’s self-esteem (i.e. is not directed negatively at their character) – so feedback from a computer had a higher positive impact than from a human teacher

Feedback is less effective when:

  • students were lacking confidence in their response in the first place, and it turns out to be incorrect.
  • it focuses on praise about the student’s character traits (“Your intelligence really comes through here….”)
  • it is made public. 64% of students prefer praise being quiet and private.
  • less confident students have their shortcomings identified – deciphered as being caused by external factors, or seen as widespread amongst the peer group.
  • feedback is poorly presented or if a student’s knowledge/understanding is insufficient to incorporate it into their subsequent actions.
  • it is too frequent, leading to student dependency on a teacher’s judgement rather than developing their own self-driven goal-setting: “Simply providing more feedback is not the answer, because it is necessary to consider the nature of the feedback, the timing, and how a student “receives” this feedback (or, better, actively seeks the feedback). As already noted, students can bias and select feedback information. The ways and manner in which individuals interpret feedback information is the key to developing positive and valuable concepts of self-efficacy about learning, which in turns leads to further learning. Teachers need to view feedback from the perspective of the individuals engaged in the learning and become proactive in providing information addressing the three feedback questions and developing ways for students to ask these questions of themselves. Students, too often, view feedback as the responsibility of someone else, usually teachers, whose job it is to provide feedback information by deciding for the students how well they are going, what the goals are, and what to do next”.

What is absent from the summary of studies is information about the age-range of students who were having their feedback impact measured, from what base level of feedback experience, over what period of time and how long the improvement resulting from feedback was sustained. Hattie and Timperley’s review of feedback studies does acknowledge certain variables :

Cultural factors: different cultures receive and prefer feedback differently: “students from collectivist cultures (e.g., Confucian-based Asia, South Pacific nations) preferred indirect and implicit feedback, more groupfocused feedback, and no self-level feedback. Students from individualist cultures (e.g., the United States) preferred more direct feedback particularly related to effort, were more likely to use direct inquiry to seek feedback, and preferred more individual focused self-related feedback”. (Susan Cain in ‘Quiet…. the power of the introvert’ writes of the more introvert cultures of Asian societies and how students who move to the more extrovert culture of the USA find the difference in teaching, learning and self-evaluation styles unsettling).

Age-related difference: older students may interpret public praise negatively in the sense of ‘so – you had low expectations of me then’, whereas criticism after failure generates a sense that the teacher had high expectations of them that they failed to reach. Yet the reverse is true for younger students in that public praise can raise motivation levels and criticism deter further effort.

Feedback focus: Hattie distinguishes between feedback that focuses on the task information (FT), that which considers the learning procedures & processing techniques (FP), that which considers the student’s learning habits and ability to self-check and self-regulate their learning (FR), and feedback aimed at the student’s character and personality (FS). Proposing that FS is least effective (whether praise or criticism), of the other three he suggests that simple, straightforward FT is better than complex; FP is more effective than FT for deeper learning; and that the aim of a comprehensive feedback process should be to move students from requiring FT, on to receiving FP and ultimately FR so they are able to self-direct and self-evaluate their learning. Within each focus stage maximum benefit arises when:

FT is focused on correcting inaccuracies in knowledge and/or misconceptions than a lack of understanding. In terms of timing it is best if it is rapid – before frustration with the task occurs.

FP is most effective when it provides cues for searching and strategising by the student. It can detract from the learning if it is too immediate. Delayed feedback has greater impact.

FR is moderated by the degree to which a student is committed to the learning goals. Positive feedback can increase motivation for a task students ‘want to do’ but such feedback actually decreases motivation for tasks they ‘have to do’. (Bit of a minefield, there then, in a mixed classro0m). In timing, the more complex the task, the more effective it is if it’s delayed.

So what are we left with as most effective feedback practice? A couple of myth-busting quotes from Hattie & Timperley:

“Simply providing more feedback is not the answer, because it is necessary to consider the nature of the feedback, the timing, and how the student ‘receives’ this feedback (or, better, actively seeks the feedback).” (p. 101)

“Feedback in not ‘the answer’; rather it is but one powerful answer. With inefficient learners it is better for a teacher to provide elaborations through instruction than to provide feedback on poorly understood concepts.” (p. 104)

The Questions this poses

Supposing that feedback remains one of the most significant vehicles of progress in learning (and that is questioned by the EEF quote and also by David Didau in this critique of ‘effect-sizes’ and their statistical validity here), what implications am I left with? :

  1. Feedback can have a negative impact for students who have acquired insufficient knowledge or lack confidence in their ability to achieve goals. Time would be better spent on ensuring they have the requisite knowledge and/or trying to establish viable goals to which they can commit, rather than giving them feedback. Who, in my classes, does this apply to?
  2. Feedback is dismissed by the student who doesn’t commit to the goal. There is no ‘gap’ to be closed. Or they self-select an alternative goal that may not involve learning. I’m increasingly of the opinion that many students who are routinely assigned ‘aspirational’ targets based on FFTD or 4 levels of progress – simply dismiss these. But school accountability systems designed to show inspectors the rigour of the progress data rarely acknowledge a capacity to change them to something more credible in the students’ eyes and re-engage their sense of a purpose.
  3. Feedback can have a deletrious impact if it is too positive in some circumstances. Commited students can interpret it as meaning my expectations of them are too low.
  4. Feedback can be too frequent. There are categories of feedback, and some of them have more impact if they are delayed.
  5. Feedback is the ‘silver bullet’ of the moment to determine quality of teacher performance. Previously it has been ‘sharing learning objectives’, ‘group discussion’, ‘multi-part lesson plans’. They didn’t crush the light out of teachers’ eyes. Feedback is the ‘must see’ totem now – but probably won’t be in four years’ time; something else will have replaced it as it didn’t do what it promised. Only, by then, we will have seen some valued colleagues pack their bags – not with marking – and depart for good.

I recognise that I could be liable to confirmation bias in my interpretation of these studies. I am hugely concerned at the increase in marking load that has been expected of teachers over the last 18 months. It is greater than I have ever known before and it is driven by inspection fears. I know of too many teachers, in too many schools (whether in RI or rated Good – it is the same) where the expectation of more frequent, more detailed marking and feedback routines have become excessive. And I’m convinced the impact is neither as great as is commonly supposed, in the direction that is anticipated, or is sustainable. I see, and read too often of, teachers losing the light in the eye from this daily, weekly, monthly requirement. I am too aware of some who will pack it in because there is so little respite.

Feedback and marking are too important not to get right. They underpin so much of what we do, should do, and build upon. But that’s why it’s so important to have a sophisticated approach to them so that we get them right, at the appropriate times, in the most suitable format for students of different ages, stages and predicaments. There isn’t a one-size-fits-all answer to this question, but the route of enquiry is being squeezed out by some pretty unsubtle checks and expectations. Two approaches I really warm to are those at Saffron Walden County High School, as described by Tom Sherrington @headguruteacher whereby each department is experimenting with a ‘closing the gap’ policy that best suits their own subject. It reeks of intelligence and trust. And it’s just what students need – not the cold hand of a whole-school policy that deadens students with each encounter they are prevailed upon to endure, but something that will vary throughout their day, demanding they ask different questions of themselves and their work.

http://headguruteacher.com/2012/11/10/mak-feedback-count-close-the-gap/

The other is by Amy Ladbrook @trueenglish and the subtle categories of different marking styles seems to embody what Hattie is alluding to with his range of Feedback categories:

http://trueenglish365.wordpress.com/2013/06/18/8-4-the-self-directed-feedback/

I know feedback is most effective under some of the conditions described above. My Year 13s, Year 12s in the months prior to their AS, at GCSE – finally – Year 11s, they show those characteristics that Hattie defines: they are aware of their goals (those grades they finally know they need for the next stage they wish to commence), they are FRers – they regulate their own study, ask for past papers to be marked and actively seek out feedback. You see them devour the advice and come back hungry for more. I’m not so convinced that marking the work of every student who passes before me each week is likely to benefit to the same extent. And those who come in looking for ‘evidence of students making substantial progress as a response to regular feedback’ are likely to be disappointed. Some students, at some stages, with some of their work – yes. And I’ll try to extend it to more. But for many of my students the most effective use of marking will not be in the direct impact on their progress, but in my understanding of them as individuals, as idiosyncratic learners – for my own differentiation practice. For others, the best use of ‘marking time’ will not be in the marking of their work, but sitting down with them and either equipping them with the knowledge and information they require before any feedback can be initiated, or discussing with them goals that they can willingly subscribe to and which will create a ‘gap’ worth attempting to ‘close’.

Would that feedback, marking and assessment could initiate progress where progress can be made, grant me differentiation provision that is informed and well-advised, (yes – there will be limited visible evidence of that too – you will have to trust me that I’m doing it), and that all of it be sustainable and give my faculty a work-life balance on a full teaching load.

That’s a bit of a prayer for survival.

Boxer didn’t make it to the end of the book.

Advertisements

17 Comments

Filed under Feedback, Ofsted

17 responses to “What if Feedback wasn’t all it was cracked up to be.

  1. A very interesting blog.

    So, we think we know that feedback is, in general, good. Intuitively, this feels right (from our own experiences, at least). However, we also know that the delivery detail of the feedback is important (again from our own experience, at least).

    The research available does not appear to clearly tell us what feedback should look like in specific circumstances (e.g. depending on age of student). Any attempt to be prescriptive would, therefore, appear to be premature.

    I wonder what work has been done to investigate effectiveness of feedback with respect to the context of that feedback? For example, how important is it that the students understand what the teacher/school/feedback is attempting to achieve? And, given the rush to feedback, is enough attention being given to this aspect?

    • That’s the tightest summary of a blog that could be managed. Yes – to the statements. To the question – context is usually at the heart of the most effective answers; a thorough understanding of it and the variations of response that might emerge from it. The 3 things that (should) take most time outside the classroom are marking, planning and creating new, more effective resources. Concerned that the last of these is being pushed out by the increased demands of the other two. And if the marking is not as efficacious as we are led to believe – then, in sum, we’re having a reduced impact. Thanks for reading and commenting.

  2. A very interesting blog. In my experience it is the culture of trust where dialogue between children and adults is meaningful that enables feedback to improve outcomes.

    • Yes! Wanted to go into that too but risked the blog becoming too lengthy. Much of the positive effect of marking is developing that culture of trust in the autumn term through an effective dialogue that can be done by going through books/folders. One of the most beneficial effects of more frequent marking over the last 12 months, is getting to know the way the students think and operate more effectively. That has virtue in all sorts of ways – differentiating, establishing targets but probably most importantly – as you note – developing a sense of trust so that each student starts to buy in to the objectives I can start setting for them. So, lots of value – but all that is ‘invisible’ to a work/feedback scrutiny check. Thanks for commenting Alison.

  3. Pingback: Forcefed: Why reducing feedback might increase learning | David Didau: The Learning Spy

  4. Pingback: Marking | Pearltrees

  5. Pingback: What if Feedback wasn't all it was cracked up t...

  6. Diana

    I really liked the post and it got me thinking about how much time teachers spend correcting papers and the importance of the feedback. Qualified Scandinavian language teachers launched some innovative and simple programs aimed at English teachers that reduce time spent on correcting papers while improving the quality of the feedback.
    If you’re interested in hearing about it, please contact me and I will introduce you to the programs.

  7. Pingback: No room for martyrdom | Ramblings of a Teacher

  8. Pingback: #feedbackandmarking | ajgingell

  9. Pingback: What if you marked every book, every lesson? Reinventing the feedback wheel | Improving Teaching

  10. Pingback: Growing Great Teachers – Which research group? | Penn Wood CPD

  11. Pingback: Maths Marking and Feedback Decisions | Penn Wood CPD

  12. Pingback: The Marking | thesequietvoices

  13. Pingback: Marking, Feedback and “Closing the Gap” Policy | mrbenney

  14. Pingback: Marking; planning for feedback- identifying then closing the gap. My #TLT session | mrbenney

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s