OECD’s Ahelo project could transform university hierarchy

The UK must decide by 31 May whether to take the next step in a project aiming to measure learning outcomes at a global level

May 7, 2015
Source: Paul Box/Report Digital
Major adjustments: the Ahelo project ‘would just open up so many possibilities to have a more level playing field’ on a global level

The world’s richest nations are due to decide this month whether to press ahead with a major international project to measure university teaching quality that could offer a “fantastic opportunity” for East Asian universities to improve their standing, possibly at the expense of the Western elite.

The Organisation for Economic Cooperation and Development’s Assessment of Higher Education Learning Outcomes (Ahelo) project is seen as the university-level equivalent of the OECD’s highly influential global Programme for International Student Assessment tests for school pupils – and as having the potential to transform the established hierarchy in world higher education.

After completion of an Ahelo feasibility study in December 2012, the OECD has asked member nations to indicate by 31 May whether they wish to take part in a full “main study”.

Turn right for a ‘teaching REF’

In the UK, the Conservative election manifesto commits to introducing a framework to recognise universities offering the “highest teaching quality” – which would rely on some measure such as Ahelo being created.

Andreas Schleicher, the OECD’s director for education and skills, told Times Higher Education that, globally, Ahelo “would just open up so many possibilities to have a more level playing field that is less influenced simply by past reputation and those kinds of things”.

Mr Schleicher predicted that Ahelo could mean that “you will see institutions that may not be as expensive, that may be a bit in the shadow, but delivering superior learning outcomes”.

He also said: “If you think of those elite universities, maybe much of their advantage is actually derived from social background: the intake they take, the selection. We have no idea currently.”

The OECD does not intend to produce a ranking of institutions using the results. But Mr Schleicher predicted that others would do so using available results. “I can’t imagine that these results would not be used, and shouldn’t be used, in a way that ranks institutions,” he said.

The OECD describes the Ahelo feasibility study as having looked at the scientific and practical feasibility of a test of what students know and can do at graduation.

In February this year, Greg Clark, the UK’s universities and science minister, chaired a meeting of nations interested in pursuing Ahelo, where the proposal for a main study was discussed.

Conclusions from that meeting included that the tests in a full scheme should focus on “transferable skills” such as critical thinking or problem solving.

Mr Schleicher said that it would take a “critical mass” of eight to 10 OECD member nations supporting the project if it were to be taken forward. “Some countries are very supportive of the work, others a lot less,” he added.

A jump-start for East Asia

A “critical mass” of a nation’s universities joining the scheme is also considered necessary to make the project work. Mr Schleicher continued: “The countries most strongly in favour of this are the countries that have most to gain, like the countries in East Asia, who basically have no chance to compete on traditional rankings which are valuing history and tradition and past reputation. For them, this is a fantastic opportunity to make a jump-start and put themselves on the map.”

He also said: “What I credit UK policymakers for is that they may see a short-term loss for them, but they see a long-term gain, in terms of you can rely on past reputation only for so long. If you want to remain competitive in the long run, you have to deliver superior quality.”

But he added of the UK government: “Whether they get their institutions along, that’s a big question.”

David Willetts, the UK’s former universities minister, said that there was currently a “frustration…that such a high part of university assessment is based on research and the prior attainment of their students”.

He added: “So my starting point is this [Ahelo] is absolutely the correct agenda and it’s really important for students that they know how universities are doing on the teaching side. And indeed it’s also reflected in the Conservative manifesto.”

However, Alison Wolf, Sir Roy Griffiths professor of public sector management at King’s College London, said that Ahelo is “unrealistic” and “will not provide governments with any useful information”.

“The courses people take in universities are hugely diverse: it is not like basic schooling where everyone is doing some common content,” she added.

The UK’s position on Ahelo will be decided when a new government is in place.


Times Higher Education free 30-day trial

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please Login or Register to read this article.

Reader's comments (2)

I cannot see how you could realistically test learning outcomes in tertiary education, given its heterogeneity. PISA tests 15-year-olds, i.e. pupils finishing basic education. In tertiary education the variability is much higher, especially regarding subject-mix. Pure and applied subjects are different, soft and hard subjects are different as well. In addition, the OECD already runs a programme to assess the skills of the adult population - PIAAC. With its data, we can already compare the skills of graduates in different countries. It is hard to see what AHELO would add, and it would hardly be a cost-effective programme. Its costs would be huge.
That's an interesting development to observe from a Brazilian perspective. In order to assess its universities (and especially the private, for-profit ones), Brazil created already in 2007 a national exam that measures students' performance. Or rather: how well the universities educated the students. Every year, certain academic areas are tested and participation is compulsory for students in their last year. This year, its the turn of management, administration and design: http://portal.inep.gov.br/enade. The results are anonymized and publicly available. Even though the Brazilian Ministry for Education is explicit about ENADE not being a ranking, universities use their result frequently in advertising. Having said all this, I would be curious to learn if and how the OECD considered the Brazilian experience in developing AHELO.