The best way to deal with the flawed journal peer-review system, says Abbott Katz, is to print everything and judge later
Let's end the dithering over journal peer review. Let's turn the system on its head. Let's print everything.
I'm quite serious. If the critics of peer review have it right - if the system really is riddled with caprice and chronic discord among reviewers, spite-driven rejections and even occasional idea thievery by reviewers - then we should start afresh. Once journals migrate in force to the web, let them hand out parcels of server space to each and every submission streaking across their transoms. But don't think peer review would evaporate as a result. It wouldn't. Rather, the model I'm espousing would vest the reviewer's calling in journal readerships instead.
You may wonder if lifting the editorial trammels off the review process would do nothing more than roll out the academic imprimatur for voodoo scientists, post-modernist poseurs and Philistines of all persuasions, freeing them to stuff their CVs with vast tufts of intellectual excelsior.
But that shouldn't happen. The system I envision would indemnify itself through a complement of checks and controls that, with a modicum of due diligence, would uphold peer review and perhaps even tweak the process salubriously.
Thus an all-comers-accepted online journal might sport the following kinds of feature: a full-text search engine a la JSTOR, and a pair of page counters displayed alongside every article entry, to count reader accesses of each piece of work. All articles would be heralded by an abstract, to which one of the counters would be assigned. Readers advancing to the article in full would be recorded by the second counter. The two access figures would distinguish casual scanners, for whom the abstract might suffice, from readers who would care to essay the entire article, and thus register a more meaningful hit.
Readers, paying or otherwise, could plumb journal contents via an e-mail address sign-in; but the hit counters would refuse to tally any non-institution-based address, thus applying the sieve to Hotmail or Yahoo signatories such as Aunt Eunice or cousin Rex who might be conscripted by desperate authors craving to swell their access scores. The counters would likewise ignore authors' accesses of their own pieces, for the same reason.
Those qualifications aside, the "hit" data would transmit a real-time commentary on the popularity - and presumably the worthiness - of submissions.
In addition, an automated citation index would bore through electronic journals, counting references to all articles; these data would likewise attach to each entry. And to complete the feedback loop, signed reader comments would be linked to each article (one assumes churlish comment-makers would be recognised as such, redounding to their own discredit).
These measures would in sum weave an evaluative gestalt - a telling, if not definitive, judgment about each article. Charlatans craving the thrill of seeing their name in print would incur the ignominy they deserve, both from infinitesimal hit counts as well as hard-eyed readers crouching in the bushes, their slings and arrows levelled at suspect articles. Potential employers would be able to repel dross-laden CVs, once the rarely read, oft-disparaged publications trumpeted therein are tracked back to their journal sources.
In fact, this new certainty of publication might impose a healthy prudence on submitters, as the only thing standing between authors and their send button would be their gifts for self-vetting and for earmarking wise and helpful editor/colleagues. After all, how many serious practitioners of their craft, envisaging the slow descent of their CV into the abyss that swallows bad work, would knowingly loose their B-grade material on their fellows?
This, then, is true peer review, and not some anarchic flight-from-standards simplism a first reading might allege. As framed here, peer review remains very much integral to the process but is situated in the post-publication phase. Remember that the potential readerships - and reviewers - of which we speak are no dilettantish horde; they are professionals who, one must assume, bring some measure of steadfastness and seriousness of purpose to their work and to the work of others. And if the whole idea still seems a mite too vox populi - if one believes that the aggregate discernment of sociologists or historians or physicists cannot be trusted to thresh good from bad - then what are we really saying about their fields?
So let's publish everything, and let the invisible hands do their thing. Let all due approbation devolve upon the worthy, and let mass opprobrium beset the wanting. In graduate school, I was told most work manages to get published eventually. I don't know if that estimate jibes with the editorial facts but, by guaranteeing publication to all, all get a hearing. Then let peers decide.
Wouldn't quality prevail?
Abbott Katz is a lecturer in sociology at MST College, London.