'Serious defects' apparent in 'crude' European rankings project

U-Map and U-Multirank fail to solve problems they aimed to fix, study suggests. Hannah Fearn writes

June 24, 2010

A major European Union-funded project attempting to overcome the problems of "validity" and "rigour" in university rankings has fallen at the first hurdle, according to a paper published this week.

The European Commission has launched two new ranking projects, U-Map and U-Multirank, aimed at reflecting the diversity of universities.

The paper, drawn up for the League of European Research Universities by Geoffrey Boulton, senior honorary professorial Fellow in the School of Science and Engineering at the University of Edinburgh, says that both initiatives have "serious defects".

University Rankings: Diversity, Excellence and the European Initiative concludes that the two projects, which map out the activities carried out by universities and rank them by their performance in those areas rather than via a single "monotonic" table, fail to tackle the problems they were created to solve.

Professor Boulton identifies two key problems with ranking systems: that they seek to capture characteristics that cannot be directly measured, and that the method is inappropriate because universities fulfil very different roles from one another.

These dilemmas have not been addressed by the EU-funded projects, he argues.

"They suffer from imprecise proxies and the profound difficulty of finding comparable data between countries," the paper states.

It claims that the exercise could become a "populist trick" in which universities are forced to carry out "food labelling" of their work, as if they were producing a consumer product.

Nevertheless, Professor Boulton's study welcomes the "attempt to create U-Map as a description of diversity" and supports the development of U-Multirank as a means of "exploring its potential to mitigate the problems of other systems".

Rankings have become the "antithesis of the university ideal", he warns.

"It's about measuring a brand, like Gucci or Chanel. It almost paints universities as a fashion accessory."

The paper claims that U-Multirank will suffer because statistics gathered from different countries are not comparable, and that there is the potential for "gross" misinterpretation.

Professor Boulton says it was inevitable that different tables produced by the European Commission would be drawn together into a single ranking, regardless of the intentions.

Criticising the trend for ranking institutions as "nonsensical" and "crude", he says: "We don't have a very clear view of what ranking is for and what it tells us.

"If I started producing a paper in which my objectives were unclear and my proxies varied from the spurious to the highly hypothetical, then my paper simply wouldn't be accepted."

The Times Higher Education World University Rankings also come under fire for including a survey of academic reputation.

"Such approaches are most likely to reinforce existing, conventional stereotypes," the paper says.

The study argues that all rankings have the perverse effect of forcing universities to behave in the same way. It finds that 980 universities across Europe had proposed to achieve a high level of international excellence in research - "an unrealistic expectation".

"If they are poor measures, and yet remain powerful drivers of behaviour, there is a serious risk of delivering more damage than benefit," it concludes.

hannah.fearn@tsleducation.com

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Have your say

Log in or register to post comments

Featured Jobs

Assistant Professorship in Behavioural Science LONDON SCHOOL OF ECONOMICS & POLITICAL SCIENCE LSE
Foundation Partnerships Officer LONDON SCHOOL OF ECONOMICS & POLITICAL SCIENCE LSE

Most Commented

James Fryer illustration (8 September 2016)

Some lecturers will rightly encourage forms of student interaction that are impossible for those covering their faces, Eric Heinze argues

University of Oxford students walking on campus

University of Oxford snatches top spot from Caltech in this year’s World University Rankings as Asia’s rise continues

Handwritten essay on table

Universities must pay more attention to the difficulties faced by students, says Daniel Dennehy

Theresa May entering 10 Downing Street, London

The prospect of new grammar schools on the horizon raises big questions for HE, writes Nick Hillman

Nosey man outside window

Head of UK admissions service Mary Curnock Cook addresses concerns that universities might ‘not hear a word’ from applicants