Scholars ‘not yet convinced by metascience movement’

Metascience researchers face distrust and suspicion from many scientists despite the discipline’s growing support among ministers and research funders, a report has found.

In the study by the Research on Research Institute (RoRI), based on insights from 18 countries, researchers dedicated to examining how research funding and selection processes can be improved explain that their findings are often met with “resistance” by scholars committed to the principle of research excellence through traditional scientific peer review.

Drawing on interviews with researchers in several countries, the study describes how enthusiasm for metascience at government level is not always matched by institutional or discipline-level support.

The RoRI report, titled Seize the Meta-Moment, analysed institutions, investment and infrastructure dedicated to metascience and examined nearly 500,000 publications globally. In it, interviewees from Australia explained how grant reviewers were sometimes “actively hostile to metascience research”, a resistance that they ascribed to “anxiety about funding research that will scrutinise their discipline”.

In Brazil, authors described a “soft rigidity” in the resistance to metascience that “arises from the culture of the scientific community itself”. Although researchers “increasingly recognise” the insights from metascience, the report claims, they “often resist translating these insights into changes in the existing system of values, evaluation criteria, indicators, and institutional practices”.

In Germany, metascience advocates find their evidence “frequently acknowledged but only selectively integrated into decision-making, as traditional peer-review hierarchies are slow to adapt”.

In the US, where the National Science Foundation has recently established a dedicated Office of Metascience, one scientist based at the US National Institutes of Health explained resistance to the nascent discipline, noting how metascientists “don’t seem to be terribly interested in feedback from us”.

Other parts of the research community are worried that the new Office for Metascience could be used by US president Donald Trump to interfere in research funding decisions. One public administration scholar claimed its role would be “political activity, not…scientific activity” and the “evaluation office…needs to be an office that’s independent and trusted”.

In Korea, scientists feared that the introduction of new ways of working would simply “increase administrative burdens to researchers”. Meanwhile, an Indian academy leader quoted in the report defended expert judgement against global rankings and institutional metrics, stating: “We don’t therefore trust any of the numbers in any case. We only trust the peer review system and nothing else.”

Other countries beyond the US have also invested significantly in metascience, with the UK pledging to spend £49 million on a cross-governmental Metascience Unit, and Research Ireland will launch a meta-research unit next month.

However, some metascience researchers worried that political support for their discipline might not last, with some complaining that they lacked institutional support because their work was regarded as peripheral to the core research of their departments.

In Brazil, several interviewees claimed that they were regarded as “a solitary Don Quixote figure advancing an agenda largely alone”. In Australia, metascientists said that “where their work had influenced Australian research policy, it has done so through personal relationships rather than institutional channels”.

In the UK, metascience was sometimes described as a “side hustle” alongside researchers’ main jobs. “Testimony…was consistent”, explained the report, with the verdict that “it is not yet viable to build a UK academic career in metascience from scratch”, with those investing time in the discipline describing the field as “a very high-risk business”.

The 265-page report also pointed to the growing support for the discipline dedicated to how science evaluates and improves itself.

James Wilsdon, lead author of the study and executive director at RoRI, said: “Metascience is having its moment: political attention, new money and a shared vocabulary have arrived together, from Washington to London; Beijing to Dublin; Tokyo to the Hague. But moments pass,” he said.

“The real test is not how much apparatus we can build while attention lasts – it’s whether we embed the data, relationships and habits of experimentation that will be useful when it moves on.”

jack.grove@timeshighereducation.com