Research assessment in New Zealand could be marked down

A review following the latest iteration of the PBRF could lead to radical changes that undermine the gains made, warns Roger Smyth

May 14, 2019

The results of New Zealand’s six-yearly assessment of university research performance, the Performance-Based Research Fund (PBRF), were released on 30 April. Assessment panels studied evidence portfolios compiled by about 8,000 academics across New Zealand’s eight universities and rated the research performance of each.

As in the three previous PBRF rounds, the universities of Auckland and Otago, which house New Zealand’s two medical schools, were assessed as having the largest number of staff producing high-quality research. Between them, Auckland and Otago employ 53 per cent of the university staff who earned an A grade (meaning that they are deemed world class). As a result, they capture half the quality assessment funding distributed on the basis of the PBRF, amounting to NZ$315 million (£160 million) a year.

But that is only part of the story. In assessing research performance, scale matters. Auckland is by far New Zealand’s biggest university by any measure: revenue, staff or student numbers. But controlling for staff numbers, Wellington’s Victoria University ranks first, meaning that it has the highest concentration of research staff and the lowest proportion of staff who didn’t meet the standard for being rated. Otago rates second and Auckland fourth.

Controlling for student numbers, meanwhile, sees Lincoln University (New Zealand’s smallest university by every measure) rank first, with Otago second and Auckland third.

The release of the results from this fourth round captured less public attention than earlier rounds, with few stories making the non-education media. This is partly because, 16 years on, the PBRF has lost its novelty. It also reflects the fact that it is no longer possible to rank universities by subject, reducing the scope for them to boast.

This change from previous rounds is one consequence of a major review of the PBRF conducted in 2012-13 in response to longstanding complaints about the exercise’s high transaction costs and the opportunities it presented for universities to “game-play”. But despite the resulting simplification of the system – not to mention the tweaks following the evaluations that followed the first two PBRF iterations – the government has now commissioned yet another review.

Already under way, this is the result of a manifesto commitment made by the incoming Labour-led coalition government in 2017. Radical change – or complete abolition – of the PBRF has long been an article of faith for New Zealand’s Tertiary Education Union (TEU), and its officials worked hard in the run-up to the election to make sure that the Labour party knew its view.

In response to the latest results, TEU president Michael Gilchrist reiterated the union’s complaints about compliance costs, despite the system’s simplification. He acknowledged that the PBRF had increased the focus of academics on research but added that “after four gruelling rounds of individual assessments those benefits are well past”. He also complained that the refocusing had come at the expense of teaching and had damaged locally focused research.

The union, in addition, has always been concerned that the PBRF assesses each individual academic staff member’s research. Many staff have argued that the “unit of analysis” in any research assessment should be research groups or academic departments – or anything other than the individual staff member.

But reputation and prestige are powerful motivators and there is little doubt that making the individual staff member the unit of analysis changed the focus of most academics’ work – making research a priority for many. Ministry of Education analysis shows that, as a result, the introduction of the PBRF saw an increase in the quality and quantity of research. New Zealand universities boosted both their share of the world’s premium publications and their share of global citations. Those gains increased, moreover, as the PBRF bedded in. And, whatever the union might argue, there is no evidence of any fall-off in the quality of teaching.

The complaints and the lobbying have continued unabated, however. Will the new review lead to the complete overhaul sought by the TEU? And if so, will that come at the expense of rising research performance? Watch this space.

Roger Smyth is an independent consultant and adviser. He is the former head of tertiary education policy at New Zealand’s Ministry of Education.

后记

Print headline: Tough tests can boost research

登录 或者 注册 以便阅读全文。

请先注册再进行下一步

获得一个月的无限制地在线阅读网站内容。只需注册并完成您的职业简介.

注册是免费的,而且非常简单。一旦成功注册,您可以每个月免费阅读3篇文章。:

  • 获得编辑推荐文章
  • 率先获得泰晤士高等教育世界大学排名相关的新闻
  • 获得职位推荐、筛选工作和保存工作搜索结果
  • 参与读者讨论和公布评论
注册

相关文章

欢迎反馈

Log in or register to post comments

评论最多

Recent controversy over the future directions of both Stanford and Melbourne university presses have raised questions about the role of in-house publishing arms in a world of commercialisation, impact agendas, alternative facts – and ever-diminishing monograph sales. Anna McKie reports

3 October