Examiners give hugely different marks

Study finds inconsistencies in grading for borderline 2:1 and 2:2 essays

April 30, 2015

Source: Rex

Grading gamble: the paper says marking is a ‘judgement, not a measurement’

Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests.

As part of the study into inconsistent marking, 24 academics in four disciplines were recruited from 20 UK universities and each asked to mark five pieces of student work originally graded as borderline 2:1 or 2:2 by universities.

But many of the academics – who had all worked as external examiners – ended up awarding hugely different marks, according to the study published in the journal Assessment and Evaluation in Higher Education.

Some essays were given a first, while others received a third-class mark, says the study, which is titled “Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria”.

Indeed, very few of the assessors in history, chemistry, nursing and psychology believed that the students’ assignments were borderline, with only seven placing all the work within two adjacent grade bands.

Nine of the 20 assignments were also ranked both best and worst by different examiners, it says. Only one of the pieces was ranked the same by all six assessors, it adds.

The publication comes midway through a fundamental review of quality assurance by the Higher Education Funding Council for England, in which the role of external examining has again come under scrutiny.

Madeleine Atkins, its chief executive, has said that she supports external examining but wants to see the process strengthened.

A review in 2011 by Dame Janet Finch, former vice-chancellor of Keele University, found that the process was working well, although concerns have been raised at how different academics apply the same criteria to ensure that academic standards remain broadly comparable across the sector. The paper argues that “a large element of unreliability” would always remain in assessment, despite efforts to moderate work, as grading is a “judgement, not measurement”.

“We should recognise the impossibility of a ‘right’ mark in the case of complex assignments, and avoid over-extensive, detailed, internal or external moderation,” it adds.

Sue Bloxham, emeritus professor in academic practice at the University of Cumbria, who co-authored the paper, said that, except for new markers, we should stop using moderation to achieve the “correct” mark for each assignment because it probably does not exist.

“We spend considerable time moderating these individual assignments, whereas fair and reliable awards are obtained because students are judged on a profile of work assessed by a range of academics,” Professor Bloxham told Times Higher Education. “Moderation should be retained only for heavily weighted items such as final-year dissertations, and that should be really thorough because of the impact on students’ awards,” she added.

The report concludes: “Universities need to be more honest with themselves, and with students, and help them to understand that application of assessment criteria is a complex judgement and there is rarely an incontestable interpretation of their meaning.”

jack.grove@tesglobal.com

Times Higher Education free 30-day trial

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (5)

'Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests'. This, of course, implies that the person who 'under-marks' is wrong. Why not express this as concern at 'over-marking'? Strangely, no student has ever come to me and said 'I think you might have marked me too highly, could you take it down a couple of points? Prof Tarr or Dr Feather would have'... ... ...
Forget conventional grading of essays. Instead, have all students read each other's. All through the term. Ask that they refer to each other in all essays after the first -- refer to issues others have raised, examples they've given, language they've wonderfully used. At the end of the term the final writing asks students to name the top influences from other students through the term. This will not be a mere list, but a chance to open up and credit others very specifically. The instructors job in grading is only to count how many full-paragraph or otherwise substantive credits each student got from the others. Those with the most thanks from others get the highest grades.
Phil, students find it difficult reading their own essays before submission!! what chance they would read others??
My daughter is in pieces after being very harshly marked on first two assignments whilst a friend who got very similar marks last year got firsts in hers but marked by the different lecturers, she now wants to give up. What can I do
Nobody wishes to see a student 'in pieces' Jan; but on which criteria are you concluding that your daughter has been 'harshly marked'?

Have your say

Log in or register to post comments

Featured Jobs

Reader in Politics and Policy

St Marys University, Twickenham

Engineer

Cern

Professor of Anthropology

Maynooth University

Preceptor in Statistics

Harvard University

Postdoctoral Fellowship in Electrochemistry

Norwegian University Of Science & Technology -ntnu
See all jobs

Most Commented

Doctoral study can seem like a 24-7 endeavour, but don't ignore these other opportunities, advise Robert MacIntosh and Kevin O'Gorman

Matthew Brazier illustration (9 February 2017)

How do you defeat Nazis and liars? Focus on the people in earshot, says eminent Holocaust scholar Deborah Lipstadt

Improvement, performance, rankings, success

Phil Baty sets out why the World University Rankings are here to stay – and why that's a good thing

Laurel and Hardy sawing a plank of wood

Working with other academics can be tricky so follow some key rules, say Kevin O'Gorman and Robert MacIntosh

Warwick vice-chancellor Stuart Croft on why his university reluctantly joined the ‘flawed’ teaching excellence framework