New online tool seeks to correct biases in student evaluations

Researchers have built an online tool that allows university teachers to assess how “hidden biases” may be affecting their student evaluation scores, as universities increasingly rely on such scores in hiring and promotion decisions.

Developed by scholars Yashar Bashirzadeh, Luc Meunier and Robert Mai, the “set bias corrector” is based on a dataset of more than 377,000 student-professor pairings collected over more than a decade across business school campuses.

The researchers launched the tool after carrying out a study that looked at how nationality, gender and age can influence student evaluation scores, which are often used by universities to assess faculty performance and to inform decisions about promotions.

Their research found that these scores can systematically advantage or disadvantage certain groups. One of the most striking findings relates to “cultural distance”, defined as the difference between the backgrounds of students and professors. The results show that professors who are more culturally distant from their students tend to score higher in student evaluations.

“What we find is that if you have a diverse faculty, the learning experience is more enriched. Diversity is something that the students appreciate,” Bashirzadeh, an assistant professor of marketing at the Grenoble École de Management, told Times Higher Education.

He added that the effect remains even after accounting for a wide range of factors. “I am taking into account factors specific to the professor, factors specific to the student…and this effect is still highly significant,” he said.

The researchers also found that younger female professors receive lower evaluations than their male counterparts. Scores for women rise towards the mid-career point before dropping again. Male professors typically see their scores decrease with age.

“For example, if you’re a young female professor who is British, and you’re teaching a British audience, you have a low cultural distance. We know there is a bias against young female professors,” Bashirzadeh explained. “So if you get a teaching evaluation score of 3.5 out of 5, in reality you might have received a 4.2 if you were a young male professor teaching a mixed audience.”

The set bias corrector applies these statistical patterns to estimate what a professor’s evaluation score would look like if certain characteristics were different. When users enter information such as age and gender, the tool generates an adjusted figure intended to reflect an unbiased score.

“We know that the bias exists. So we can take your score and ask, ‘What would you have received if one of those characteristics were different?’” said Bashirzadeh.

He stressed that the findings on cultural distance are salient at a time when policies related to diversity and internationalisation are increasingly coming under threat, pointing to examples such as Donald Trump’s attacks on DEI initiatives at US universities and a push for classes to be taught in Dutch in the Netherlands.

“Our findings show that diversity is something that students value,” he said.

Bashirzadeh also warned against treating faculty diversity as a simple binary between “international” and “domestic”.

“If you’re British and you hire someone from Ireland, that’s very different from hiring someone from say Germany or Brazil,” he said. “You have different degrees of cultural distance.”

seher.asaf@timeshighereducation.com