Examiners give hugely different marks

Study finds inconsistencies in grading for borderline 2:1 and 2:2 essays

April 30, 2015

Source: Rex

Grading gamble: the paper says marking is a ‘judgement, not a measurement’

Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests.

As part of the study into inconsistent marking, 24 academics in four disciplines were recruited from 20 UK universities and each asked to mark five pieces of student work originally graded as borderline 2:1 or 2:2 by universities.

But many of the academics – who had all worked as external examiners – ended up awarding hugely different marks, according to the study published in the journal Assessment and Evaluation in Higher Education.

Some essays were given a first, while others received a third-class mark, says the study, which is titled “Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria”.

Indeed, very few of the assessors in history, chemistry, nursing and psychology believed that the students’ assignments were borderline, with only seven placing all the work within two adjacent grade bands.

Nine of the 20 assignments were also ranked both best and worst by different examiners, it says. Only one of the pieces was ranked the same by all six assessors, it adds.

The publication comes midway through a fundamental review of quality assurance by the Higher Education Funding Council for England, in which the role of external examining has again come under scrutiny.

Madeleine Atkins, its chief executive, has said that she supports external examining but wants to see the process strengthened.

A review in 2011 by Dame Janet Finch, former vice-chancellor of Keele University, found that the process was working well, although concerns have been raised at how different academics apply the same criteria to ensure that academic standards remain broadly comparable across the sector. The paper argues that “a large element of unreliability” would always remain in assessment, despite efforts to moderate work, as grading is a “judgement, not measurement”.

“We should recognise the impossibility of a ‘right’ mark in the case of complex assignments, and avoid over-extensive, detailed, internal or external moderation,” it adds.

Sue Bloxham, emeritus professor in academic practice at the University of Cumbria, who co-authored the paper, said that, except for new markers, we should stop using moderation to achieve the “correct” mark for each assignment because it probably does not exist.

“We spend considerable time moderating these individual assignments, whereas fair and reliable awards are obtained because students are judged on a profile of work assessed by a range of academics,” Professor Bloxham told Times Higher Education. “Moderation should be retained only for heavily weighted items such as final-year dissertations, and that should be really thorough because of the impact on students’ awards,” she added.

The report concludes: “Universities need to be more honest with themselves, and with students, and help them to understand that application of assessment criteria is a complex judgement and there is rarely an incontestable interpretation of their meaning.”

jack.grove@tesglobal.com

Times Higher Education free 30-day trial

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (5)

'Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests'. This, of course, implies that the person who 'under-marks' is wrong. Why not express this as concern at 'over-marking'? Strangely, no student has ever come to me and said 'I think you might have marked me too highly, could you take it down a couple of points? Prof Tarr or Dr Feather would have'... ... ...
Forget conventional grading of essays. Instead, have all students read each other's. All through the term. Ask that they refer to each other in all essays after the first -- refer to issues others have raised, examples they've given, language they've wonderfully used. At the end of the term the final writing asks students to name the top influences from other students through the term. This will not be a mere list, but a chance to open up and credit others very specifically. The instructors job in grading is only to count how many full-paragraph or otherwise substantive credits each student got from the others. Those with the most thanks from others get the highest grades.
Phil, students find it difficult reading their own essays before submission!! what chance they would read others??
My daughter is in pieces after being very harshly marked on first two assignments whilst a friend who got very similar marks last year got firsts in hers but marked by the different lecturers, she now wants to give up. What can I do
Nobody wishes to see a student 'in pieces' Jan; but on which criteria are you concluding that your daughter has been 'harshly marked'?

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Monster behind man at desk

Despite all that’s been done to improve doctoral study, horror stories keep coming. Here three students relate PhD nightmares while two academics advise on how to ensure a successful supervision

celebrate, cheer, tef results

Emilie Murphy calls on those who challenged the teaching excellence framework methodology in the past to stop sharing their university ratings with pride

Sir Christopher Snowden, former Universities UK president, attacks ratings in wake of Southampton’s bronze award

Reflection of man in cracked mirror

To defend the values of reason from political attack we need to be more discriminating about the claims made in its name, says John Hendry

But the highest value UK spin-off companies mainly come from research-intensive universities, latest figures show