Variability in marking ‘too high’, says study

Author of new paper on grading reliability says universities need to tackle ‘pernicious’ issue 

January 23, 2017
people going up escalator
Source: Alamy

Universities must address the variability between lenient and tough graders to ensure students do not make “dysfunctional” academic decisions during their studies, according to the author of new research on the subject.

In his paper “The relationship between grading leniency and grading reliability”, Ido Millet found that when grading leniency goes up, grading reliability – the correlation between students’ grade point average and the grades they receive in that class – goes down. Professor Millet, professor of management information systems at Pennsylvania State University’s Behrend College, said the variability between lenient and tough graders is a “pernicious issue” in higher education.

"The difference between the most lenient departments and the toughest departments is about 0.45 on the negative side and 0.5 on the positive side. That’s a huge gap: it’s like the gap between a B and an A,” he told Times Higher Education. “[And] these are by department, which have at least 100 course sections. So the numbers are significant.

“Academia needs to move to reduce that variability because it causes dysfunctional decisions by students. Students might be biased to pick majors, minors and electives where higher grades are easier to achieve.”

Professor Millet’s study looked at data from more than 50,000 course sections at a North American university over several years. He found that variability did not occur only within particular departments, but also among individual faculty members.

“This variability again is dysfunctional. Students may just go for the more lenient faculty members, more lenient majors to graduate in,” he said. “I don’t think it’s a question of [students thinking]: ‘I’m going to fail here and I’m going to succeed here. It’s just that I’m going to work harder here and get lower grades.’ It’s a double whammy.”

The paper suggests that if the number of academic faculty who mark leniently reaches “critical mass” in a given department, other instructors “might be inclined to conform to lowered grading norms” because of “student and peer pressure”.

Professor Millet urged universities to develop strategies to combat the variability between tough and lenient markers to avoid these repercussions.

One of his proposals would be to report on leniency variability across faculty members in a “sensitive manner” – to raise staff awareness – and collect information about the number of assignments students are required to undertake and the amount of time they take to complete. In cases where less is required of students, the information would “push faculty” to do more assessment and assign more work.

“Those faculty that…teach more and have larger class sizes may sacrifice some of the reliability of their grading both [in] how much energy they put into the grading, as well as [in] the number of assignments they give students,” he said.

“If we start collecting data like how many hours [students] spend on [a] course outside of class per week, that would also go a long way to pushing grading reliability higher.” 

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Reader's comments (1)

Hmm... Another headline based on the US. Is it the case in the UK? Certainly not a significant issue in the Engineering department where I work.

Have your say

Log in or register to post comments

Featured Jobs

Most Commented

Monster behind man at desk

Despite all that’s been done to improve doctoral study, horror stories keep coming. Here three students relate PhD nightmares while two academics advise on how to ensure a successful supervision

celebrate, cheer, tef results

Emilie Murphy calls on those who challenged the teaching excellence framework methodology in the past to stop sharing their university ratings with pride

Sir Christopher Snowden, former Universities UK president, attacks ratings in wake of Southampton’s bronze award

Reflection of man in cracked mirror

To defend the values of reason from political attack we need to be more discriminating about the claims made in its name, says John Hendry

But the highest value UK spin-off companies mainly come from research-intensive universities, latest figures show