OECD may fill ‘missing link’ in world rankings

Learning outcomes project aims to get to grips with international teaching comparisons. John Morgan reports

February 1, 2010

An international study will measure and compare learning outcomes at universities around the world – and could provide the “missing link” for university rankings.

The Organisation for Economic Co-operation and Development (OECD) launched the Assessment of Higher Education Learning Outcomes (AHELO) initiative in Washington on January.

AHELO’s first phase will measure students’ “generic” skills, such as problem solving and critical thinking, through online tests adapted from the US’ Collegiate Learning Assessment.

The questions are intended to be non-specialised and answerable by undergraduates of all disciplines.

Fourteen nations are expected to take part in the full project, including the US, Finland, Italy, Japan and Mexico.

The goal is for all 30 OECD countries to participate eventually.

Richard Yelland, head of the education management and infrastructure division of the OECD’s Education Directorate, who is leading the initiative, said: “AHELO is a pioneering international attempt to assess the quality of higher education by focusing on what students have learnt during their studies and what skills they have acquired. Success will provide higher education systems and institutions with diagnostic tools for improvement that go far beyond anything currently available.”

Mr Yelland told The Chronicle of Higher Education: “This isn’t going to be a ranking. It is so much more. If we manage to produce reliable data, some people may well turn them into rankings, but that is not what this is about.”

For the 14 nations taking part in the full project, an average of 10 institutions per country will be involved, with about 200 students per institution taking part.

In the US, institutions in four states – Connecticut, Massachusetts, Missouri and Pennsylvania – are participating.

Ben Wildavsky, author of a forthcoming book on the globalisation of higher education, The Great Brain Race, described AHELO as “a very promising initiative”.

He added: “It has the potential to provide the missing link in existing university assessments and rankings, which have often been faulted for overemphasising research, neglecting undergraduate education and focusing on input measures at the expense of rigorous measures of student-learning outcomes.”

Mr Wildavsky, senior fellow in research and policy at the Ewing Marion Kauffman Foundation in Kansas City, a body that promotes entrepreneurship, added: “If the OECD can pull it off, it will have filled a major gap and moved us towards a world in which universities have much better incentives to improve an important part of their work – teaching students.”

Mr Wildavsky said the involvement of the US, with support from its Department of Education, was key.

However, there will be “technical barriers to coming up with truly valid, reliable and comparable tests across nations”, he added.

“I also feel that the best measures of university effectiveness would include not only student learning but also research prowess – but perhaps that will come later if and when AHELO proves its worth,” he said.

Phil Baty, editor of the Times Higher Education World University Rankings, said: “One of the biggest weaknesses in the current global rankings systems is that there is nothing that can fairly measure teaching quality across different countries and cultures, so the AHELO project is being watched very closely indeed. But inevitably it will be very difficult and will take a long time before there are any serious data for detailed comparisons between institutions.

“In the meantime, Times Higher Education is working closely with its new rankings partner, Thomson Reuters, to make sure that there are more sensible and rigorous measures of teaching quality for the World University Rankings in 2010 and beyond. We are working on a far more sophisticated reputational survey of world academics, where we expect teaching quality to be unpicked in a way that’s never been done before in world rankings.”


Already registered?

Sign in now if you are already registered or a current subscriber. Or subscribe for unrestricted access to our digital editions and iPad and iPhone app.

Have your say

Log in or register to post comments

Featured Jobs

Register to continue  

You've enjoyed reading five THE articles this month. Register now to get five more, or subscribe for unrestricted access.

Most Commented

  • Boats docked in Port Hercule, Monaco

Richard Murphy praises a bold effort to halt tax-dodging by the 1 per cent

It’s a question with no easy answer, finds James Derounian

  • James Fryer illustration (19 November 2015)

With no time for proper peer review and with grade inflation inevitable, one academic felt compelled to resign

  • Worker checks thin-film silicon solar module, Truebbach

Asia doubles representation while European countries face varied performance

  • Lisa Mckenzie, Class War Party candidate, Chingford

Anarchist academic reflects on what her recent brush with the law says about threats to academic freedom