World’s university ‘oligopoly’ accused of blocking OECD bid to judge learning quality

Ahelo academic and funder blame research elite for thwarting international graduate tests, but criticisms also levelled at multimillion-dollar ‘failure’

August 6, 2015
Bison on road, Yellowstone
Source: Getty
Disruption: one academic said that established institutions were seeking to ‘prevent new information about education coming to light’

Attempts to measure what students learn at different universities around the world are being thwarted by the “established oligopoly” of institutions seeking to “prevent new information about education coming to light”, according to an academic who worked on plans for a standardised test.

The Organisation for Economic Cooperation and Development appears to have hit the buffers in its bid to persuade member states to support the introduction of its Assessment of Higher Education Learning Outcomes tests for graduates, seen as the university-level equivalent of its Programme for International Assessment tests in school education.

Andreas Schleicher, OECD director for education and skills, had previously suggested that such a test could challenge the established hierarchy of world higher education by creating measures of teaching quality not based on past reputation, potentially offering a particular boost to institutions in East Asia.

Hamish Coates, chair of higher education at the University of Melbourne, who led the OECD’s feasibility study on Ahelo, told Times Higher Education: “Broadly what’s happening is the established oligopoly isn’t keen to be disrupted by the new order, the new commercial order, of higher education.

“What we’re seeing happen is essentially the legacy providers, who are very dominant in the research space, trying to prevent new information about education coming to light.”

Earlier this year, the OECD asked member nations to say by 31 May whether or not they wanted to start a full-scale Ahelo project, following the feasibility study. The OECD says that no final decision has yet been reached. But as THE revealed last month, the Westminster government has confirmed that England will not take part, and many observers believe that the OECD will be forced to abandon the scheme, which is reported to have cost $13 million (£8.3 million) in its development up to 2013.

The American Council on Education, which represents about 1,700 higher education institutions, and Universities Canada wrote a joint open letter in May to Angel Gurría, the OECD secretary-general, in which they outlined “grave reservations” about the project and criticised its “exorbitant costs”.

Ahelo, “which attempts to standardize outcomes and use them as a way to evaluate the performance of different institutions, is deeply flawed”, they said.

But Professor Coates said that the average parent around the world wishing “to send their child to higher education wants to know about what the cost, quality and prospects of higher education are for their son or daughter”.

He also argued that “information about the quality of learning is absolutely essential to having even a discussion – let alone an institutional or a national policy – around improving higher education”.

Professor Coates noted that steps had been taken to develop similar measures within nations, including studies of learning gain commissioned by the Higher Education Funding Council for England and projects funded by the German federal government. “We do see this work progressing, albeit on a national basis,” he added.

His view on opposition to Ahelo was echoed by Dewayne Matthews, vice-president of strategy development at the Lumina Foundation, a US body that aims to increase the proportion of Americans with degrees and provided $750,000 to the OECD for Ahelo.

Dr Matthews said that “higher education is not all that interested in being externally accountable for its results, to be blunt”.

Asked why Lumina had funded Ahelo, he replied that the foundation’s “work on increasing attainment in the US has led us inevitably to questions of the quality of credentials”.

Dr Matthews argued that opposition to Ahelo was led by “the more selective institutions”, whose graduates are in demand among employers. Those selective institutions did not want scrutiny on the question of whether the value of their graduates came from “value added” by the institution’s education rather than from the institution’s selection criteria, he suggested.

But Philip Altbach, director of the Centre for International Higher Education at Boston College, said most observers see Ahelo as a “failure”, adding that it was “nonsense” to claim that elite universities had blocked the project “because they fear losing their prominence…While it is the case that current rankings emphasise research – because research is all that can reasonably accurately be measured – few would argue that these [elite] institutions have poor teaching quality.”

He added: “Measuring learning outcomes, or teaching quality for that matter, across [different] nations and cultures would in my view be quite difficult and perhaps impossible.”


Print headline: Bid to judge learning blocked by ‘oligopoly’

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please Login or Register to read this article.

Related articles

Reader's comments (2)

If anyone were to have reservations about such an attempt it has to come from two directions. 1. What is being measured and how? We already know that the REF in the UK is completely distorting research behaviour to the detriment of good long term research, so a teaching equivalent is bound to have a similar effect. 2. The statement by Prof. Coates reflects a very anglo-saxon view of HE, one where fees are paid for universities and there is a focus on parents wanting to know about the cost of their children's education and whether it is worth it. Perhaps we should start by considering the overall purpose of a university within society as whole and thus assess how they contribute in a multiplicity of ways rather than start another measuring narrowly focussed measuring exercise. Finally, any such exercise will inevitably have a homogenising effect on teaching content and methods around the world as institutions battle it out to climb (or avoid falling) in any such league tables. It will lead to an even more depressing world in higher education.
It is curious to read that "few would argue that these [elite] institutions have poor teaching quality.” Some can do this quite well! In the US there is the possibility to read it based on solid research, such that published in 'Academically Adrift: Limited Learning on College Campuses', which includes those 'elite institutions'. The same authors confirm their findings of... let's say 'poor assessment practices' in 'Aspiring Adults Adrift: Tentative Transitions of College Graduates'. Therefore, I find difficult to understand why Prof Altbach thinks that is impossible to argue that teaching quality is not consistent across the field. Research data and analysis on grade inflation are also opening an avenue to discuss how teaching and assessment of learning outcomes stay consistent with official quality standards. An expedient "nonsense” is not replacing an academic debate on this issue; it is also not clear what exactly is difficult to assess across cultures when higher education across the world is based on the same historic model. Where exactly is the insurmountable difficulty of measuring learning outcomes in higher education?