Astuteness of crowds

November 4, 2010

Many believe that above a certain threshold of competence, the award of research council grants has become almost a lottery ("Grant award? It could be you", 30 September).

The success rate is very low; many applicants have little confidence in the review system; and researchers suspect that the quality of the reviewers has gone down, with many experienced experts too busy to examine the ever-increasing number of applications.

With just a few reviewers, one or two adverse comments, a throwaway line in a review or the unlucky assignment of your mortal enemy to your application, months or years of work can be ruined.

I suggest an alternative. Instead of a couple of experts giving detailed reviews, each application should be sent electronically to as many reviewers as possible. Each reviewer must merely grade each application from one to 10; more detailed comments would be optional. A computer would then rank the applications by arithmetical means (no need for expensive and distorting council panels) and award the applications from the top mean down until the cash ran out.

We know from work on the wisdom of crowds that pooling many judgements can deliver highly accurate estimates of underlying quality. My system would be faster, cheaper, fairer and more accurate.

Trevor Harley, Head of the School of Psychology, University of Dundee.

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Register
Please Login or Register to read this article.

Sponsored