Lack of student success measures is holding back higher education

From undergraduate engagement to teacher training and graduate salaries, universities lack statistics needed to track student outcomes, say Hamish Coates and Xi Gao

November 1, 2021
graphic of student data illustrating op-ed about 9 points to consider in measuring student success
Source: istock

Whatever else the pandemic may have done, it has put education and learning firmly on university leaders’ agendas. Almost every university has engaged in complex education transformation, while the associated socio-economic shocks have sharpened employers’ and governments’ interest in the production of advanced skills.

But one thing that has been conspicuously lacking from the debate has been informative and robust information on education – and, in particular, on student learning. This is kind of weird, as it is almost the first thing people who have little to do with higher education would expect us to have lying around.

We know from two decades of research, spanning dozens of countries and 10 per cent of the world’s universities, that it matters how learners engage with peers and teachers. It matters how they challenge themselves, use resources and participate in high-impact activities. But applicants, students, teachers and employers would all benefit from nine key pieces of information on education and learning.

First, there is a need to match radically different forms of provision with today’s growing and diversifying learner markets. Rather than “admit people to courses”, there is a growing need to “link needs with resources”. One issue is whether faculties, institutions and systems offer learner/learning-centric, rather than structural, systems to those who need them. An important indicator of learning surely boils down to whether platforms are in place that enable people with opportunities to engage.

Institutional infrastructure also matters, though in unfolding and interesting ways. The volume of library books, ivy-covered buildings, plush endowments and silver-coated research reputations are not sufficient information about value. As education experiences increasingly happen online and later in life, universities’ investments in digital infrastructure, education partnerships and hybrid learning spaces matter a lot.

However, too few indicators of university performance touch on expenditure. This is partly due to the opaque nature of university accountants and partly due to complexities around generalisability. But, realistically, resources cannot be ignored. In particular, the relative budget spent on education services – as opposed to research, works or operations – affirms an institution’s investment in engaging students. Obviously, such information must be relativised to the kind of institution in question.

Fourth, data do exist about the support provided by institutions. But what support meant in the 1990s is different from what is needed today and over the next 20 years. Ultimately, moreover, it is the impact rather than provision of supports that counts. Do institutions have the capacity to understand and help each student where they are? How long does it take an institution to know a student has missed a milestone, and then deploy an effective response? This requires platforms and experts capable of implementing appropriate forms of information, analytics, diagnostics and interventions.

Fifth, we should also avoid fixating on student engagement with the formal “curriculum”. The reality is that learners build up knowledge using a tapestry of supplied and found information. So meaningful questions about student engagement focus on whether institutions are furnishing situated resources that are accessible to non-traditional students, can be co-created and involve experience of work as well as the local and global communities.

Indeed, as the world steers towards sustainability and myriad forms of co-contribution, even the most prestigious pockets of higher education are moving towards co-creation with a suite of communities. This turns the concept of “enrichment” on its head. Rather than students being “enriched” within an institutional frame, what matters more is whether the institution is reaching beyond its walls and enriching the lives of much broader, potentially global populations. This spotlights university commitment to sustainability goals.

Seventh, looking into a future in which higher education institutions will employ dedicated teachers, as opposed to teaching/research uber professors, it also becomes meaningful to ask about teaching capability and integrity. As higher education expands and threats to standards multiply, ensuring quality requires increasing complexity and infrastructure. How many faculty are trained and developed as teachers? To what extent do student-facing faculty engage in continuous professional development? How much time do teachers have to spend on students? How is academic integrity being sustained? What teaching innovations is the institution making?

Eighth, we must remember that most people engage with higher education for personal success. For most, this means getting a job or doing higher value work. Recent years have seen a proliferation of econometric indicators of graduate returns and returns on public investment, but the ultimate question is whether cogent information is available to help stakeholders and participants alike understand – and potentially improve – the value higher education creates.

Ninth, we should not lose sight of the fact that higher education success goes beyond financial reward. It is also reasonable to require information on learners’ and institutions’ engagement with broader forms of success. Do they even know what they are meant to achieve in this regard? Are institutions engaged in the kinds of assessment validation and innovation required to assure success? Do they report generalisable measures of student success?

Clearly, these nine indicators shift from looking at how students participate within institutions to looking at how people and communities engage in hybrid ways to co-create learning opportunities and credentials. Luckily, lots of the data are lying around to flesh out these topics. The information just needs to be brought together. This will play a major role in building knowledge, transforming skills and shaping the post-pandemic future.

Hamish Coates is a tenured professor at Tsinghua University’s Institute of Education and director of the Higher Education Research Division. He will be speaking at Times Higher Education’s Teaching Excellence Summit, held in Tianjin, China, from 16 to 18 November. Xi Gao is a doctoral student at Tsinghua’s Institute of Education.

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Please Login or Register to read this article.

Related articles

Related universities

Reader's comments (3)

It is certainly not the case in the UK that there is a lack of data collection. However, what makes academics disillusioned is that no visible use is made of the results. It would be useful to see some feedback on the difference made by the statistics, which were not collected when I was a student or when I took my first academic post.
You need to collect the right sort of data. Too much emphasis is put on 'Student Voice' - subjective comments from learners who generally prefer a class that is an 'easy ride' to one that is challenging and hard, and who often use surveys to comment on personal likes/dislikes of their teachers or to air generic grievances. Engagement and grades give a better picture of how effective the teaching is than 'did they enjoy this course', especially if you can track improvement over the course of a student's enrolment, not just a snapshot at the end.
In my own opinion, the use of the "copy-n-paste" way of supposedly having to have studied something, and then submitting that as an answer, is the reason for so many "graduates" not knowing how to do what they presented themselves as knowing. I've experienced way too many times when someone claimed to have a Masters degree in a subject, yet couldn't answer some the simple of questions, although they felt that they deserved to be awarded the title of Manager or Supervisor. Wait, do what.? They can't do the work themselves, but yet they want to manage or supervise the workers.? WOW......