Arts experts fear metrics will skew REF

Researchers are anxious that quality indicators could become benchmarks, reports John Gill

July 24, 2008

Leading figures in the arts, humanities and social sciences have reinforced concerns about plans to use statistical indicators to assess research quality and allocate funding.

Shearer West, director of research at the Arts and Humanities Research Council, and Onora O'Neill, president of the British Academy, both underlined the fears that still exist about the role of metrics in the new research excellence framework.

The REF, which is to replace the research assessment exercise to determine the allocation of billions of pounds a year in research funding, is to include a combination of peer review and statistical indicators, such as the number of times an academic's published work is cited.

At a conference on research funding last week, Professor West said that "the spectre of bibliometrics arouses more anger and anxiety than just about any of the myriad forms of assessment we have been subjected to since the 1980s". She said that metrics posed conundrums in the arts and humanities that did not apply to science subjects.

The first of these, she said, was linked to the breadth and range of subjects covered by the arts and humanities, and to the huge number of journals in the field, which are both single discipline and interdisciplinary in the themes, periods and countries they cover.

"The creation of new journals is part of engendering new debates, disciplines and innovation. Relying on well-established journals in a narrow range of disciplines would fossilise research," Professor West said.

A second area of concern is that journals are not the "gold standard" in the majority of subjects in the arts and humanities - monographs are still internationally valued outputs in the humanities, as are performances, exhibitions and "created outputs" in the arts.

"While one might imagine a time in which monographs could be added to citation indices, ephemeral outputs create huge technical challenges," she said.

Citation habits in the arts and humanities "mean the number of citations does not provide a stable indicator of quality or impact. There is also a widely held anxiety that indicators can quickly become targets and, when this happens, behaviour becomes distorted," she said.

Professor West said there was now an acceptance by those developing the REF that statistical indicators could not be used alone to assess research quality. "However, we are moving through a metric age, and it will not help us in the arts and humanities to refuse to engage in the debate. Numbers can provide us with a useful reference point, providing we use them carefully and always subject them to the human judgment of experts," she said.

Baroness O'Neill, speaking at the British Academy's annual general meeting, said there was still "widespread concern" about the REF proposals, warning that the greater use of metrics may lead to "unintelligent forms of accountability".

Referring to last year's British Academy report on peer review, she said it "did not conclude that metrics are never useful, but noted their tendency to alter behaviour and argued that they should be used only with caution and to augment, rather than replace expert judgment".

john.gill@tsleducation.com.

登录 或者 注册 以便阅读全文。

请先注册再进行下一步

获得一个月的无限制地在线阅读网站内容。只需注册并完成您的职业简介.

注册是免费的,而且非常简单。一旦成功注册,您可以每个月免费阅读3篇文章。:

  • 获得编辑推荐文章
  • 率先获得泰晤士高等教育世界大学排名相关的新闻
  • 获得职位推荐、筛选工作和保存工作搜索结果
  • 参与读者讨论和公布评论
注册

欢迎反馈

Log in or register to post comments

评论最多

Recent controversy over the future directions of both Stanford and Melbourne university presses have raised questions about the role of in-house publishing arms in a world of commercialisation, impact agendas, alternative facts – and ever-diminishing monograph sales. Anna McKie reports

3 October