Examiners give hugely different marks

Study finds inconsistencies in grading for borderline 2:1 and 2:2 essays

April 30, 2015

Source: Rex

Grading gamble: the paper says marking is a ‘judgement, not a measurement’

Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests.

As part of the study into inconsistent marking, 24 academics in four disciplines were recruited from 20 UK universities and each asked to mark five pieces of student work originally graded as borderline 2:1 or 2:2 by universities.

But many of the academics – who had all worked as external examiners – ended up awarding hugely different marks, according to the study published in the journal Assessment and Evaluation in Higher Education.

Some essays were given a first, while others received a third-class mark, says the study, which is titled “Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria”.

Indeed, very few of the assessors in history, chemistry, nursing and psychology believed that the students’ assignments were borderline, with only seven placing all the work within two adjacent grade bands.

Nine of the 20 assignments were also ranked both best and worst by different examiners, it says. Only one of the pieces was ranked the same by all six assessors, it adds.

The publication comes midway through a fundamental review of quality assurance by the Higher Education Funding Council for England, in which the role of external examining has again come under scrutiny.

Madeleine Atkins, its chief executive, has said that she supports external examining but wants to see the process strengthened.

A review in 2011 by Dame Janet Finch, former vice-chancellor of Keele University, found that the process was working well, although concerns have been raised at how different academics apply the same criteria to ensure that academic standards remain broadly comparable across the sector. The paper argues that “a large element of unreliability” would always remain in assessment, despite efforts to moderate work, as grading is a “judgement, not measurement”.

“We should recognise the impossibility of a ‘right’ mark in the case of complex assignments, and avoid over-extensive, detailed, internal or external moderation,” it adds.

Sue Bloxham, emeritus professor in academic practice at the University of Cumbria, who co-authored the paper, said that, except for new markers, we should stop using moderation to achieve the “correct” mark for each assignment because it probably does not exist.

“We spend considerable time moderating these individual assignments, whereas fair and reliable awards are obtained because students are judged on a profile of work assessed by a range of academics,” Professor Bloxham told Times Higher Education. “Moderation should be retained only for heavily weighted items such as final-year dissertations, and that should be really thorough because of the impact on students’ awards,” she added.

The report concludes: “Universities need to be more honest with themselves, and with students, and help them to understand that application of assessment criteria is a complex judgement and there is rarely an incontestable interpretation of their meaning.”

jack.grove@tesglobal.com

Times Higher Education free 30-day trial

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (5)

'Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests'. This, of course, implies that the person who 'under-marks' is wrong. Why not express this as concern at 'over-marking'? Strangely, no student has ever come to me and said 'I think you might have marked me too highly, could you take it down a couple of points? Prof Tarr or Dr Feather would have'... ... ...
Forget conventional grading of essays. Instead, have all students read each other's. All through the term. Ask that they refer to each other in all essays after the first -- refer to issues others have raised, examples they've given, language they've wonderfully used. At the end of the term the final writing asks students to name the top influences from other students through the term. This will not be a mere list, but a chance to open up and credit others very specifically. The instructors job in grading is only to count how many full-paragraph or otherwise substantive credits each student got from the others. Those with the most thanks from others get the highest grades.
Phil, students find it difficult reading their own essays before submission!! what chance they would read others??
My daughter is in pieces after being very harshly marked on first two assignments whilst a friend who got very similar marks last year got firsts in hers but marked by the different lecturers, she now wants to give up. What can I do
Nobody wishes to see a student 'in pieces' Jan; but on which criteria are you concluding that your daughter has been 'harshly marked'?

Have your say

Log in or register to post comments

Most Commented

United Nations peace keeper

Understanding the unwritten rules of graduate study is vital if you want to get the most from your PhD supervision, say Kevin O'Gorman and Robert MacIntosh

David Parkins Christmas illustration (22 December 2016)

A Dickensian tale, set in today’s university

Eleanor Shakespeare illustration (5 January 2017)

Fixing problems in the academic job market by reducing the number of PhDs would homogenise the sector, argues Tom Cutterham

poi, circus

Kate Riegle van West had to battle to bring her circus life and her academic life together

Houses of Parliament, Westminster, government

There really is no need for the Higher Education and Research Bill, says Anne Sheppard