One of the strongest criticisms of our old methodology for the World University Rankings was that 40 per cent of the score for each institution was based on a survey of a very small number of academics, writes Phil Baty.
The so-called peer-review score for the 2009 rankings was based on poll responses from around 3,500 individuals, with responses aggregated over three years to give a total sample size of just 9,386.
Given the millions of academics worldwide, it is not surprising that this aspect of the methodology evoked the most ire: the weighting for peer review was simply too high and the sample size was too low. Drilling down to subject level, the numbers were tiny.
So we are delighted to confirm that for the 2010 Times Higher Education World University Rankings, our new rankings partner Thomson Reuters has commissioned one of the world's leading polling companies, Ipsos Mori, to carry out research to support the peer-review element of the tables.
Using a professional polling company means that we can inject proper targeting and transparency into the process while ensuring that we get a much larger response rate than in the past - the aim is for at least 25,000 responses.
It also means that the questions in the opinion survey can be carefully crafted to elicit meaningful and consistent responses while ensuring that every respondent knows what is being asked of them.
Heads of institutions interested in joining our survey "platform group", to help us to achieve the best possible structured sample, should email me at email@example.com.
This step alone will make a dramatic improvement to the 2010 rankings - but the whole methodology is currently undergoing a root-and-branch review, so join the discussion and complete our reader survey at http://bit.ly/ErAag