Variability in marking ‘too high’, says study

Author of new paper on grading reliability says universities need to tackle ‘pernicious’ issue 

January 23, 2017
people going up escalator
Source: Alamy

Universities must address the variability between lenient and tough graders to ensure students do not make “dysfunctional” academic decisions during their studies, according to the author of new research on the subject.

In his paper “The relationship between grading leniency and grading reliability”, Ido Millet found that when grading leniency goes up, grading reliability – the correlation between students’ grade point average and the grades they receive in that class – goes down. Professor Millet, professor of management information systems at Pennsylvania State University’s Behrend College, said the variability between lenient and tough graders is a “pernicious issue” in higher education.

"The difference between the most lenient departments and the toughest departments is about 0.45 on the negative side and 0.5 on the positive side. That’s a huge gap: it’s like the gap between a B and an A,” he told Times Higher Education. “[And] these are by department, which have at least 100 course sections. So the numbers are significant.

“Academia needs to move to reduce that variability because it causes dysfunctional decisions by students. Students might be biased to pick majors, minors and electives where higher grades are easier to achieve.”

Professor Millet’s study looked at data from more than 50,000 course sections at a North American university over several years. He found that variability did not occur only within particular departments, but also among individual faculty members.

“This variability again is dysfunctional. Students may just go for the more lenient faculty members, more lenient majors to graduate in,” he said. “I don’t think it’s a question of [students thinking]: ‘I’m going to fail here and I’m going to succeed here. It’s just that I’m going to work harder here and get lower grades.’ It’s a double whammy.”

The paper suggests that if the number of academic faculty who mark leniently reaches “critical mass” in a given department, other instructors “might be inclined to conform to lowered grading norms” because of “student and peer pressure”.

Professor Millet urged universities to develop strategies to combat the variability between tough and lenient markers to avoid these repercussions.

One of his proposals would be to report on leniency variability across faculty members in a “sensitive manner” – to raise staff awareness – and collect information about the number of assignments students are required to undertake and the amount of time they take to complete. In cases where less is required of students, the information would “push faculty” to do more assessment and assign more work.

“Those faculty that…teach more and have larger class sizes may sacrifice some of the reliability of their grading both [in] how much energy they put into the grading, as well as [in] the number of assignments they give students,” he said.

“If we start collecting data like how many hours [students] spend on [a] course outside of class per week, that would also go a long way to pushing grading reliability higher.” 

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Reader's comments (1)

Hmm... Another headline based on the US. Is it the case in the UK? Certainly not a significant issue in the Engineering department where I work.

Have your say

Log in or register to post comments

Most Commented

Lady Margaret Hall, Oxford will host a homeopathy conference next month

Charity says Lady Margaret Hall, Oxford is ‘naive’ to hire out its premises for event

Laurel and Hardy sawing a plank of wood

Working with other academics can be tricky so follow some key rules, say Kevin O'Gorman and Robert MacIntosh

Woman pulling blind down over an eye
Liz Morrish reflects on why she chose to tackle the failings of the neoliberal academy from the outside
White cliffs of Dover

From Australia to Singapore, David Matthews and John Elmes weigh the pros and cons of likely destinations

Michael Parkin illustration (9 March 2017)

Cramming study into the shortest possible time will impoverish the student experience and drive an even greater wedge between research-enabled permanent staff and the growing underclass of flexible teaching staff, says Tom Cutterham