Variability in marking ‘too high’, says study

Author of new paper on grading reliability says universities need to tackle ‘pernicious’ issue 

January 23, 2017
people going up escalator
Source: Alamy

Universities must address the variability between lenient and tough graders to ensure students do not make “dysfunctional” academic decisions during their studies, according to the author of new research on the subject.

In his paper “The relationship between grading leniency and grading reliability”, Ido Millet found that when grading leniency goes up, grading reliability – the correlation between students’ grade point average and the grades they receive in that class – goes down. Professor Millet, professor of management information systems at Pennsylvania State University’s Behrend College, said the variability between lenient and tough graders is a “pernicious issue” in higher education.

"The difference between the most lenient departments and the toughest departments is about 0.45 on the negative side and 0.5 on the positive side. That’s a huge gap: it’s like the gap between a B and an A,” he told Times Higher Education. “[And] these are by department, which have at least 100 course sections. So the numbers are significant.

“Academia needs to move to reduce that variability because it causes dysfunctional decisions by students. Students might be biased to pick majors, minors and electives where higher grades are easier to achieve.”

Professor Millet’s study looked at data from more than 50,000 course sections at a North American university over several years. He found that variability did not occur only within particular departments, but also among individual faculty members.

“This variability again is dysfunctional. Students may just go for the more lenient faculty members, more lenient majors to graduate in,” he said. “I don’t think it’s a question of [students thinking]: ‘I’m going to fail here and I’m going to succeed here. It’s just that I’m going to work harder here and get lower grades.’ It’s a double whammy.”

The paper suggests that if the number of academic faculty who mark leniently reaches “critical mass” in a given department, other instructors “might be inclined to conform to lowered grading norms” because of “student and peer pressure”.

Professor Millet urged universities to develop strategies to combat the variability between tough and lenient markers to avoid these repercussions.

One of his proposals would be to report on leniency variability across faculty members in a “sensitive manner” – to raise staff awareness – and collect information about the number of assignments students are required to undertake and the amount of time they take to complete. In cases where less is required of students, the information would “push faculty” to do more assessment and assign more work.

“Those faculty that…teach more and have larger class sizes may sacrifice some of the reliability of their grading both [in] how much energy they put into the grading, as well as [in] the number of assignments they give students,” he said.

“If we start collecting data like how many hours [students] spend on [a] course outside of class per week, that would also go a long way to pushing grading reliability higher.” 

You've reached your article limit.

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments

Reader's comments (1)

Hmm... Another headline based on the US. Is it the case in the UK? Certainly not a significant issue in the Engineering department where I work.

Have your say

Log in or register to post comments

Featured Jobs

Assistant Recruitment - Human Resources Office

University Of Nottingham Ningbo China

Outreach Officer

Gsm London

Professorship in Geomatics

Norwegian University Of Science & Technology -ntnu

Professor of European History

Newcastle University

Head of Department

University Of Chichester
See all jobs

Most Commented

men in office with feet on desk. Vintage

Three-quarters of respondents are dissatisfied with the people running their institutions

students use laptops

Researchers say students who use computers score half a grade lower than those who write notes

Canal houses, Amsterdam, Netherlands

All three of England’s for-profit universities owned in Netherlands

Mitch Blunt illustration (23 March 2017)

Without more conservative perspectives in the academy, lawmakers will increasingly ignore and potentially defund social science, says Musa al-Gharbi

sitting by statue

Institutions told they have a ‘culture of excluding postgraduates’ in wake of damning study