Academic renewal needs pressure from below and responsibility from above

The negative consequences of an excessive reliance on metrics in research assessment have been repeatedly called out in recent years. And numerous universities, funders, publishers and research organisations have signed international pledges to do better.

Initiatives such as the San Francisco Declaration on Research Assessment (Dora), the Coalition for Advancing Research Assessment (CoARA) and the Leiden Manifesto for Research Metrics are important. They gave language to concerns that many researchers already felt but struggled to express. They made it harder for institutions to claim that journal rank and crude output measures were sufficient indicators of quality. And they created openings for narrative CVs, broader evaluation criteria, and more serious discussion of research culture.

The language of openness, collaboration, integrity, mentoring and responsible assessment is now widespread. However, the limits for such high-level initiatives are becoming equally visible. The fact is that responsible assessment has entered institutional vocabulary much faster than it has entered institutional practice, and many organisations can point to formal commitments on their websites to improve their research assessment while leaving everyday incentives largely intact.

A department can praise narrative CVs while still ranking candidates by journal lists. A funder can promote responsible assessment while still rewarding predictable productivity. A publisher can speak the language of integrity while still profiting from volume. In each case, the vocabulary changes while the operating logic remains intact.

Indeed, many of the scientists most affected by the ongoing metrics obsession have barely heard of those pledges to move beyond it. What early-career researchers are familiar with is the problem those initiatives are supposed to solve. They know the continued pressure to publish before work is ready. They know the weight still carried by journal names, citation counts, grant income, institutional prestige and personal visibility.

They are told to be bold but then advised to be strategic. They are encouraged to take intellectual risks, then assessed through short-term output. They are urged to collaborate, while systems reward individual visibility.

These contradictions shape careers and character. Young researchers learn what to value by observing what is rewarded. If the system rewards speed, they learn speed. If it rewards visibility, they learn visibility. If it rewards strategic conformity, they learn to conform. The danger is that distorted incentives become normalised as professional wisdom.

Source: 

Getty Images montage

Research is a shared intellectual enterprise, not just a route to individual advancement or institutional performance. Institutions should create the conditions in which difficult questions can be pursued seriously, imaginatively and collegially.

Discovery requires time, stability, freedom, critical exchange and the willingness to pursue lines of enquiry before their significance is fully known. It requires difficult ideas to mature, experiments to fail and sceptical peers to point towards better approaches. Institutions, therefore, need the courage to support work whose value cannot always be justified through immediate outputs.

This level of trust does not remove accountability. But it requires forms of accountability suited to scholarship: serious judgement, critical scrutiny and intellectual responsibility.

Metrics are not useless, but there is a grave peril that once they become established, they usurp human judgement. Publication counts, journal names, citation indicators, grant totals, and institutional rankings may provide fragments of information, but they cannot determine the full intellectual value of a person’s work. They cannot necessarily tell us whether a researcher asks original questions, builds reliable knowledge, opens new directions, mentors responsibly, contributes to collective understanding, or works with integrity.

Once proxies dominate, they begin to shape the work itself. Researchers learn to ask whether a question can be published quickly, whether it fits a fashionable agenda, whether it can be divided into several papers, whether its novelty can be made to sound more decisive than the evidence warrants.

These choices do not always reflect personal weakness. Often, they simply reflect an accurate reading of the system scientists were cultivated into when, as doctoral candidates and postdoctoral researchers, they were pushed to publish early and often – and when they saw colleagues who did so rewarded with grants and further publications.

Source: 

Liudmila Chernetska/Getty Images (edited)

Wider sector pressures exacerbate the problem. For instance, the overly high ratio between the number of doctoral graduates and the availability of stable academic positions leads scientists to pursue research agendas whose momentum has outlasted their originality, rather than those with the most potential for long-term impact.

Universities themselves operate within structures that reward visibility, growth, competitive positioning and measurable performance. In some systems, academic institutions are increasingly governed as businesses, with research becoming an asset to be managed through targets and indicators.

Proxies allow institutions to compare units, funders to justify decisions, and governments to allocate resources. Their appeal lies in speed and apparent objectivity. Their danger lies in displacement. Over time, what is measurable comes to stand for what matters.

This is especially egregious in lower-resourced systems, where researchers are often judged by standards developed in richer nations but without comparable infrastructure or support. This can also weaken attention to local priorities and make researchers reluctant to publish in regional journals, damaging intellectual diversity.

The growth of predatory journals, conferences and paper mills is one visible consequence. These do not flourish simply because some individuals behave badly. They flourish because academic systems reward speed and volume of publication.

If academic life is to be renewed, reform must become visible in hiring, promotion, funding, mentoring, publishing, and institutional culture. And that requires much more than declarations from above. It also requires pressure from within the research community.

Specifically, it requires people willing to ask what has changed in practice – in hiring, promotion, funding, workload allocation, doctoral training and recognition – after a university signs a pledge to assess research “responsibly”. Similar questions should be asked of funders, journals and scholarly societies. And there must be a collective determination among them not to treat symbolic compliance as sufficient.

Awareness is an essential beginning. Many early-career researchers already know that something is wrong. What is often missing is a shared diagnosis, a sense of historical and institutional context, and a vocabulary that connects individual experience to systemic design.

Early-career networks, postdoctoral associations, graduate organisations, learned societies and informal scholarly communities can help create this vocabulary. They can make research assessment a collective issue rather than a private anxiety. They can discuss Dora, CoARA, and related initiatives as tools for questioning local practice. They can ask institutions to explain what reform commitments mean in practice. They can compare hiring criteria with stated values. They can document contradictions. They can press departments and societies to adopt better norms.

Collective awareness changes what institutions can ignore. A concern voiced by one person can be dismissed as complaint, but a concern voiced by many becomes harder to evade.

Nevertheless, such action must be undertaken carefully. Early-career researchers have a particular stake in improving research assessment since they will inherit the institutions now being shaped. But even if they see the contradictions of modern academic life most clearly, those with the least institutional security cannot be expected to apply the most pressure to solve problems created by institutions. Many work under precarious conditions and intense pressure.

Senior academics have greater leverage and therefore greater responsibility. These are the people who sit on the committees overseeing hiring, promotion, funding, prizes, publication and institutional advice. These are the people who write reference letters, train junior colleagues and help decide which forms of work are recognised. If institutional values are to become real, senior researchers must use these influential positions to make evaluative practice match declared principle: to demonstrate to those entering and building academic careers that the present system is not inevitable or beyond challenge.

Source: 

Getty Images montage

Of course, it is difficult even for senior academics to enact enlightened rhetoric when the wider conditions of modern research culture still pull in the opposite direction. Hiring and promotion committees, for instance, may formally endorse broad assessment, narrative CVs, or responsible metrics but still rely heavily on journal prestige and publication volume when decisions become difficult. Evaluators are often overloaded, undertrained, uncertain across disciplinary boundaries, or under pressure to defend decisions using familiar indicators.

Institutions can help by giving committees the time and training needed to read work seriously. They can make explicit how intellectual quality is judged. They can ensure that mentoring, careful scholarship, replication, data stewardship, public engagement, negative results and long-term intellectual contribution are recognised rather than treated as secondary to visible productivity.

None of this requires a new bureaucracy. Change requires attention, language and collective insistence. It needs people to ask practical questions repeatedly and publicly enough that symbolic compliance becomes uncomfortable – because it is harder to hide.

Conversations can begin modestly. Departments can compare their stated values with their assessment criteria. Societies can reconsider how they use journal prestige and impact factors in awarding grants and prizes. Early-career networks can collect examples of meaningful reform, as well as cases where reform language masks continuity.

At a minimum, institutions that claim to support responsible research assessment should be able to explain how, in practice, they judge intellectual quality, how they protect time for serious work, how they train evaluators, and how they prevent journal prestige from standing in for judgement.

Until change is thoroughly enacted, early-career researchers in particular need to survive in the system as it is now. Mentors must therefore speak more openly with them about the difference between scholarship and career strategy, helping them navigate current realities without pretending that the system is healthy – or accepting that it cannot be made well again.   

Paul B. Rainey is director of the department of microbial population biology at the Max Planck Institute for Evolutionary Biology in Plön and professor at the laboratory for biophysics and evolution at ESPCI Paris, PSL Research University Paris. He writes a Substack on academic renewal.

Puri López-Garcia is CNRS director of research at the Ecology, Systematics and Evolution Unit at Université Paris-Saclay.

Zeynep Ceren Karahan is professor in the School of Medicine at Ankara University.

Paul Williams is emeritus professor of molecular microbiology at the University of Nottingham.

Stipan Jonjić is emeritus professor at the Centre for Proteomics, Faculty of Medicine, University of Rijeka, Croatia. 

Kenneth N. Timmis is professor emeritus in microbiology at the Technical University Braunschweig, Germany.

The authors compose a task force on academic reform for the European Academy of Microbiology.