WUR 3.0: changes to our international outlook measures

We will measure outbound student exchange and take into account the population size of a country to more fairly judge internationalisation, says Duncan Ross

June 8, 2022
Young woman in international airport
Source: iStock

The Times Higher Education World University Rankings have always had an international flavour, not just because they include universities from around the world, but also in terms of the metrics we choose to evaluate.

We believe that internationalisation deeply strengthens universities. Knowledge is inherently global and the collaboration of people from different backgrounds makes for a better learning and research environment.

To help us evaluate this, we currently have three metrics:

  • Proportion of international students
  • Proportion of international staff
  • Proportion of publications with at least one co-author from an international institution

Each of these metrics is worth 2.5 per cent of the overall score, meaning that internationalisation makes up 7.5 per cent in total. It’s a small component (only industry income is smaller at 2.5 per cent), but it can make a big difference.

For the next generation of our rankings we want to consider one additional measure, and we want to think about how we can make the current measures work better.

A new measure

For several years, we have been producing a teaching-focused ranking on Japanese universities – the Japan University Rankings. As well as including metrics on the proportion of international students and international staff, this ranking examines another aspect of internationalisation – the provision of international learning opportunities for domestic students. We measure this by collecting data on the number of students on international exchange programmes (essentially a measure of outbound student exchange). The results have been positive and we would like to include this metric in the World University Rankings.

There are decisions to make when measuring this, however. How much time do students need to spend in an international setting? Does this need to be a single block, or can it be multiple smaller sessions? In the Japan University Rankings, we count both exchanges that last less than a month and exchanges that last more than a month, giving higher weighting to the latter. For the World University Rankings, we are planning to only count exchanges that last at least one month.

One short-term challenge with introducing this metric is the Covid-19 pandemic. The first edition of the World University Rankings that will use the new WUR 3.0 methodology is WUR 2024 (to be published in autumn 2023) and the international data will be based on the 2021 academic year, when international travel was still significantly disrupted. Therefore, while we will ask universities to provide data on outbound exchange students on a voluntary basis for WUR 2024, and the data will be validated and scored, the figures will be given zero weight in the final ranking calculation. We will adjust the weights in the future.

A new normalisation

We like the existing international measures, but they are not perfect. Large countries – some of which may be home to many diverse cultures – are at a big disadvantage when compared with smaller countries and regions. Students can travel hundreds of kilometres in the US and still be in the same country. That is a luxury that doesn’t exist in Luxembourg or Qatar or many smaller nations. The result: it is “easier” for universities in Luxembourg and Qatar to do well in these measures.

We have been exploring how to deal with this challenge and have decided that the best approach is a new type of normalisation that will take account of the size of the country. There are several ways of measuring a country’s size but we think the most relevant option is to use the country’s overall population. The capacity of higher education is largely guided by this number. 

A good argument could be made for using the population of 15- to 25-year-olds (the group most likely to be receiving a university education), or 15- to 65-year-olds (the group most likely to be economically active, and therefore contributing towards the funding of universities).  However, the data for age-group populations are often less reliable than overall population numbers.

There is also a question of how to use the population values. The sizes of populations of different countries varies widely – from 1.4 billion in China to 650,000 in Luxembourg. If we were to apply these numbers directly to the metric calculation, they would skew the scores far too much.

As a result, and after much experimenting, we think a better approach is to use the logarithm of the population. This moderates the impact of size discrepancies while still providing valuable insight into the positive role of international activity.

If you would like to ask any questions about these changes or offer feedback, contact us at profilerankings@timeshighereducation.com.

Duncan Ross is chief data officer at Times Higher Education.

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Register
Please Login or Register to read this article.

Related articles

Reader's comments (1)

Thank you Duncan for always keeping Qatar in your thoughts, I am sure that we have a role to play in this :) As we welcome the "normalization" of this indicator, the hope is, out of fairness, that you apply it to all indicators, including surveys. As for one small detail, Macau is 118 km2, Hong Kong is 1114 km2, Luxembourg is 2586 km2 and Switzerland is 41,285 km2 while Qatar is 11,571 km2....just sayin... On a different note, a quick filtering of results on your ranking website by % of international students shows Qatar to be in 66th position, not a strong point for it being " “easier” for universities in Luxembourg and Qatar to do well in these measures." That said, we welcome any acknowledgment of THE of the short comings in its rankings and wish for others to do the same and look forward to still do our best to maintain our score in this indicator despite/thanks to these changes.

Sponsored

Featured jobs