The same unsavoury menu

August 22, 2013

We write as a group of concerned academics who find themselves unable to resolve a difficult academic dispute and are hoping to find guidance from the community of scholarship.

Overall student satisfaction, as measured by the annual National Student Survey, is an important part of the process by which universities are ranked. These rankings are in turn used by governments and government agencies when they develop policy and determine funding allocations. Students and their parents are encouraged to use these rankings as indicators of excellence when deciding where they would like to study.

A group of undergraduates at our university recently submitted a piece of fieldwork for a social science research module. They spoke to people who were leaving two restaurants in central London, Le Gavroche and McDonald’s Family Restaurant, and asked them to rate their overall satisfaction with the food they had just eaten. The diners from McDonald’s reported an overall satisfaction rating of 91 per cent, while those from Le Gavroche reported a rating of 83 per cent. The students therefore concluded that McDonald’s was the better restaurant, serving better food.

After being double-marked and externally moderated, the assignment failed because of its faulty methodology, but the students appealed on the grounds that it could not be at fault as it was the same as the one being used to rank universities in the NSS.

How is their case to be resolved?

Names and addresses supplied

You've reached your article limit

Register to continue

Registration is free and only takes a moment. Once registered you can read a total of 6 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Register

Reader's comments (1)

You are right to find that their students methodology is flawed in this case, even though it is similar to the one used in the NSS survey. That is because the NSS methodology is flawed. The only reason why students do not hear us criticise it openly is because part of the NSS survey method is that what universities and their staff can say is severely limited, to not influence the responses. It is not unusual to find poor research methods used in commercial (or political) environments. In their studies, the students have probably heard of many such cases, and discussed their weaknesses. And so they know, if their education is successful, that on their degree they are learning techniques that are of a better standard than those used day after day, visible for example in advertising or in the newspapers. The least students should do is, having obtained their results, discuss them in relation with methodology choices. It is not enough to say "therefore Macdo is better": the students should highlight that the results are questionable, would surprise many, and seek reasons why their fieldwork has led to them. These reasons are found in their years of study: they have heard of the weaknesses of the methods available to them, they should discuss their methods with reference to the work of others to evaluate them. If they don't, and accept unquestioningly their results without discussing the role of methodology, then they have failed to understand the problem raised by empirical study which is central to the human sciences. While the argument that the method is used elsewhere may be, arguably, a reason to accept the method, there is absolutely no excuse in not discussing and investigating it once it has been applied. I would expect the NSS methodology to be discussed, questioned, and its weaknesses found and alleviated over time. But then, my expectations may be too high: would many legislators get a sociology degree?

Have your say

Log in or register to post comments

Most Commented

question marks PhD study

Selecting the right doctorate is crucial for success. Robert MacIntosh and Kevin O'Gorman share top 10 tips on how to pick a PhD

India, UK, flag

Sir Keith Burnett reflects on what he learned about international students while in India with the UK prime minister

Pencil lying on open diary

Requesting a log of daily activity means that trust between the institution and the scholar has broken down, says Toby Miller

Application for graduate job
Universities producing the most employable graduates have been ranked by companies around the world in the Global University Employability Ranking 2016
Construction workers erecting barriers

Directly linking non-EU recruitment to award levels in teaching assessment has also been under consideration, sources suggest