Support for reputation measure; respondent profiles boost confidence

The use of reputation surveys in university rankings is controversial. Many do not like the combination of subjective information with objective data and there are concerns that reputation surveys perpetuate historical preconceptions and fail to reflect the present.

But in our consultation with the sector, there was strong support for the continued use of reputation information in the world rankings. Some 79 per cent of respondents to a survey by our rankings data provider Thomson Reuters rated reputation as a “must have” or “nice to have” measure. We operate in a global market where reputation clearly matters.

Times Higher Education sought to meet this clear demand for reputation information in its 2010-11 World University Rankings and also to address the concerns. We implemented dramatic improvements to the reputation survey itself, but at the same time reduced the weight given to reputation in the rankings.

The reputation data for 2010-11 came from a worldwide Academic Reputation Survey, carried out in spring 2010 by polling firm Ipsos, commissioned by Thomson Reuters. It was an invitation-only survey of experienced academics, issued according to the global distribution of researchers, as measured by the United Nations Educational, Scientific and Cultural Organization. It was issued in seven languages and scholars were asked to rate institutions, in both research and in teaching, in their specific field, allowing them to draw on direct experience. “Action-based” questions prompted more meaningful and thoughtful responses.

We received a record 13,388 usable responses in just three months, making the survey the biggest of its kind in the world.

We promised a transparent approach. The methodology and survey instrument were published in full and this week, the thousands of academics who took part in the survey were sent a detailed report on the respondent profile. It makes reassuring reading:

• Responses were received from 131 countries

• The average respondent had been working at a higher education institution for more than 16 years

• The average respondent had published more than 50 research papers

• The largest proportion of respondents, 38 per cent, were from the Americas. Some 30.2 per cent hailed from Asia Pacific and the Middle East, with 28.3 per cent from Europe, so a good geographical spread was achieved

• Engineering and technology comprised the subject area with the highest proportion of respondents (23 per cent), followed by the physical sciences (21 per cent). Some 18 per cent worked in the social sciences, providing a reasonable spread across the disciplines

• The average respondent spent just over half of their time on research, a third on teaching and a fifth on administration.

The high quality of the research, and especially of the respondents, gives us real confidence in the results.

The survey for the 2011-12 World University Rankings will be issued shortly to an entirely new group of academics around the world, with further improvements and in a wider range of languages.

If you receive an invitation, you will be contributing to a serious and useful piece of global research if you take the 15 minutes needed to complete the survey.

Look out for it in your in-box.