Variability in marking ‘too high’, says study

Author of new paper on grading reliability says universities need to tackle ‘pernicious’ issue 

January 23, 2017
people going up escalator
Source: Alamy

Universities must address the variability between lenient and tough graders to ensure students do not make “dysfunctional” academic decisions during their studies, according to the author of new research on the subject.

In his paper “The relationship between grading leniency and grading reliability”, Ido Millet found that when grading leniency goes up, grading reliability – the correlation between students’ grade point average and the grades they receive in that class – goes down. Professor Millet, professor of management information systems at Pennsylvania State University’s Behrend College, said the variability between lenient and tough graders is a “pernicious issue” in higher education.

"The difference between the most lenient departments and the toughest departments is about 0.45 on the negative side and 0.5 on the positive side. That’s a huge gap: it’s like the gap between a B and an A,” he told Times Higher Education. “[And] these are by department, which have at least 100 course sections. So the numbers are significant.

“Academia needs to move to reduce that variability because it causes dysfunctional decisions by students. Students might be biased to pick majors, minors and electives where higher grades are easier to achieve.”

Professor Millet’s study looked at data from more than 50,000 course sections at a North American university over several years. He found that variability did not occur only within particular departments, but also among individual faculty members.

“This variability again is dysfunctional. Students may just go for the more lenient faculty members, more lenient majors to graduate in,” he said. “I don’t think it’s a question of [students thinking]: ‘I’m going to fail here and I’m going to succeed here. It’s just that I’m going to work harder here and get lower grades.’ It’s a double whammy.”

The paper suggests that if the number of academic faculty who mark leniently reaches “critical mass” in a given department, other instructors “might be inclined to conform to lowered grading norms” because of “student and peer pressure”.

Professor Millet urged universities to develop strategies to combat the variability between tough and lenient markers to avoid these repercussions.

One of his proposals would be to report on leniency variability across faculty members in a “sensitive manner” – to raise staff awareness – and collect information about the number of assignments students are required to undertake and the amount of time they take to complete. In cases where less is required of students, the information would “push faculty” to do more assessment and assign more work.

“Those faculty that…teach more and have larger class sizes may sacrifice some of the reliability of their grading both [in] how much energy they put into the grading, as well as [in] the number of assignments they give students,” he said.

“If we start collecting data like how many hours [students] spend on [a] course outside of class per week, that would also go a long way to pushing grading reliability higher.”

john.elmes@tesglobal.com 

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (1)

Hmm... Another headline based on the US. Is it the case in the UK? Certainly not a significant issue in the Engineering department where I work.

Have your say

Log in or register to post comments

Most Commented

James Fryer illustration (27 July 2017)

It is not Luddism to be cautious about destroying an academic publishing industry that has served us well, says Marilyn Deegan

Hand squeezing stress ball
Working 55 hours per week, the loss of research periods, slashed pensions, increased bureaucracy, tiny budgets and declining standards have finally forced Michael Edwards out
Jeffrey Beall, associate professor and librarian at the University of Colorado Denver

Creator of controversial predatory journals blacklist says some peers are failing to warn of dangers of disreputable publishers

Kayaker and jet skiiers

Nazima Kadir’s social circle reveals a range of alternative careers for would-be scholars, and often with better rewards than academia

hole in ground

‘Drastic action’ required to fix multibillion-pound shortfall in Universities Superannuation Scheme, expert warns