Examiners give hugely different marks

Study finds inconsistencies in grading for borderline 2:1 and 2:2 essays

April 30, 2015

Source: Rex

Grading gamble: the paper says marking is a ‘judgement, not a measurement’

Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests.

As part of the study into inconsistent marking, 24 academics in four disciplines were recruited from 20 UK universities and each asked to mark five pieces of student work originally graded as borderline 2:1 or 2:2 by universities.

But many of the academics – who had all worked as external examiners – ended up awarding hugely different marks, according to the study published in the journal Assessment and Evaluation in Higher Education.

Some essays were given a first, while others received a third-class mark, says the study, which is titled “Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria”.

Indeed, very few of the assessors in history, chemistry, nursing and psychology believed that the students’ assignments were borderline, with only seven placing all the work within two adjacent grade bands.

Nine of the 20 assignments were also ranked both best and worst by different examiners, it says. Only one of the pieces was ranked the same by all six assessors, it adds.

The publication comes midway through a fundamental review of quality assurance by the Higher Education Funding Council for England, in which the role of external examining has again come under scrutiny.

Madeleine Atkins, its chief executive, has said that she supports external examining but wants to see the process strengthened.

A review in 2011 by Dame Janet Finch, former vice-chancellor of Keele University, found that the process was working well, although concerns have been raised at how different academics apply the same criteria to ensure that academic standards remain broadly comparable across the sector. The paper argues that “a large element of unreliability” would always remain in assessment, despite efforts to moderate work, as grading is a “judgement, not measurement”.

“We should recognise the impossibility of a ‘right’ mark in the case of complex assignments, and avoid over-extensive, detailed, internal or external moderation,” it adds.

Sue Bloxham, emeritus professor in academic practice at the University of Cumbria, who co-authored the paper, said that, except for new markers, we should stop using moderation to achieve the “correct” mark for each assignment because it probably does not exist.

“We spend considerable time moderating these individual assignments, whereas fair and reliable awards are obtained because students are judged on a profile of work assessed by a range of academics,” Professor Bloxham told Times Higher Education. “Moderation should be retained only for heavily weighted items such as final-year dissertations, and that should be really thorough because of the impact on students’ awards,” she added.

The report concludes: “Universities need to be more honest with themselves, and with students, and help them to understand that application of assessment criteria is a complex judgement and there is rarely an incontestable interpretation of their meaning.”

jack.grove@tesglobal.com

Times Higher Education free 30-day trial

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (5)

'Undergraduates awarded low scores for their work may receive top marks if they are graded by another academic, a study suggests'. This, of course, implies that the person who 'under-marks' is wrong. Why not express this as concern at 'over-marking'? Strangely, no student has ever come to me and said 'I think you might have marked me too highly, could you take it down a couple of points? Prof Tarr or Dr Feather would have'... ... ...
Forget conventional grading of essays. Instead, have all students read each other's. All through the term. Ask that they refer to each other in all essays after the first -- refer to issues others have raised, examples they've given, language they've wonderfully used. At the end of the term the final writing asks students to name the top influences from other students through the term. This will not be a mere list, but a chance to open up and credit others very specifically. The instructors job in grading is only to count how many full-paragraph or otherwise substantive credits each student got from the others. Those with the most thanks from others get the highest grades.
Phil, students find it difficult reading their own essays before submission!! what chance they would read others??
My daughter is in pieces after being very harshly marked on first two assignments whilst a friend who got very similar marks last year got firsts in hers but marked by the different lecturers, she now wants to give up. What can I do
Nobody wishes to see a student 'in pieces' Jan; but on which criteria are you concluding that your daughter has been 'harshly marked'?

Have your say

Log in or register to post comments

Most Commented

question marks PhD study

Selecting the right doctorate is crucial for success. Robert MacIntosh and Kevin O'Gorman share top 10 tips on how to pick a PhD

India, UK, flag

Sir Keith Burnett reflects on what he learned about international students while in India with the UK prime minister

Pencil lying on open diary

Requesting a log of daily activity means that trust between the institution and the scholar has broken down, says Toby Miller

Application for graduate job
Universities producing the most employable graduates have been ranked by companies around the world in the Global University Employability Ranking 2016
Construction workers erecting barriers

Directly linking non-EU recruitment to award levels in teaching assessment has also been under consideration, sources suggest