Even refs dabble in a little foul play

Our RAE-driven culture is turning peer review into an unfair game, says Tim Birkhead

Peer review is the cornerstone of scientific enterprise. The entire system of academic publishing relies on reciprocal altruism: having our papers refereed and refereeing those of others in return. While not perfect, peer review is traditionally considered objective and reasonably fair. But the system is creaking under the strain. With more researchers submitting papers to more journals, editors are running out of qualified referees. The consequence of increased demand is referee fatigue, with many just saying "no".

I average one or two review requests each day and, increasingly, I find myself refusing them. If I did not, I would be working largely for the publishers rather than for my university. Understandably, journal editors are irritated by the reluctance of referees, but there are other more sinister consequences. One is that overworked academics who agree to referee may do a poor job. Another is that in their search for suitable referees, editors may start with international experts (who are too busy); move to national experts (also too busy), then to national non-experts and so on. In the end, it will involve asking undergraduates tutored by eminent scientists to referee papers, with the result that publication decisions will be made fairly randomly.

As we free fall towards a random refereeing process, prospective authors are increasingly irritated by inappropriate decisions made by referees and editors. As a way of streamlining the process, some journals pre-screen manuscripts before deciding whether they are worth refereeing. Who does the pre-screening? My guess is that, in some cases, the electronic submission system is set up in such a way as to check the author's name and a few keywords and it then rejects or accepts (to referee only) on that basis. It sounds preposterous, but a colleague submitted a manuscript (electronically) to an eminent journal on a Sunday afternoon only to get an automated reply a few minutes later telling him that the editor felt the paper was not of enough general interest and that he should try elsewhere.

It is not just journal editors or their automated counterparts that are getting tougher, referees are becoming nastier, too, further eroding the objectivity of the peer-review process. As competition for grants increases, the have-nots are lashing out at the haves. Although we might not like it, the logic is simple: someone in your field trashed your last grant application; the chances are that it is the very person whose paper you have just been given to referee.

But take heart, referees are getting nicer as well. Nastier, nicer: it sounds contradictory? Not at all. It is analogous to what evolutionary biologists call disruptive selection, where the extremes are favoured at the expense of the intermediates. Scientists have always formed coteries or groups of colleagues whom they favour and, at the other extreme, those whom they do not. As researchers are forced to decide which papers to referee, my guess is that they are more likely to agree to review those of their colleagues and/or enemies rather than unknown researchers. They then referee their colleagues' papers positively but treat their rivals' papers more harshly. If we are to retain academic standards and objectivity in this crazy research assessment exercise-driven world, journal editors - along with the rest of us - need to urgently rethink how we deal with the issue of refereeing papers.

Tim Birkhead is professor of behavioural ecology at Sheffield University.