Google Scholar’s citation errors skew h-index leaderboards

Further concerns have been raised over the reliability of Google Scholar after academics were mistakenly awarded thousands of citations that saw them top leaderboards for their discipline.

While research sleuths have repeatedly highlighted how academics can use self-citations or citation rings to increase their h-index, the rise of inaccurate listings as a result of Google Scholar’s automated process of author attributions has now been highlighted.

In some cases, this has seen Google Scholar’s subject leaderboards skewed by erroneous attributions, usually as a result of academics sharing a common surname such as Brown, Jones or Smith.

For instance, a UK-based professor of education is highly ranked in “educational leadership” research on Google Scholar, primarily because of a string of multi-authored papers on nuclear physics published between 2010 and 2021.

And an Australian education professor is credited with several highly cited outputs written by the influential US economist Jeffrey Sachs, now based at Columbia University.

Turkish scholar Selahattin Turan topped the educational leadership table but only thanks to more than 80,000 citations for a qualitative research guide, for which he was the Turkish translator. Turan said he was not aware of the “relevant errors” added by Google until he was contacted by Times Higher Education. He subsequently amended his profile.

“This is evidently a problem at Google Scholar’s end but many academics seem not to be correcting the mistakes,” said one social science professor who contacted Times Higher Education about how the “little-known issue” of erroneous citations presented a misleading picture about the leading players in a particular field.

“Some authors – those who aren’t citations- and h-index-obsessed – may be entirely unaware that their profiles are being greatly enhanced under false pretences, but others must surely be aware of the errors,” they added, noting the increased prevalence of “Google Scholar profiles containing inaccurate listings”.

Created in 2004, Google Scholar is one of the leading indexes on scientific impact, with h-indexes often used as measure of research standing. In 2014, its architect Anurag Acharya said scholars would be incentivised to correct their profiles because any inconsistencies could be easily checked.

Reese Richardson, postdoctoral fellow at Northwestern University’s Centre for Science of Science and Innovation, who studies research malpractice, noted that “authors and institutions do stand to gain from not maintaining their profiles when these mistakes occur”, even if “these instances likely arise from errors in Google Scholar’s indexing”.

“Despite Google Scholar’s ubiquity in citation-based research assessment, some academics clearly do not maintain their profiles, often to their own citation benefit,” said Richardson, who urged the internet search giant to prioritise accuracy and integrity over introducing new innovations to drive engagement.

“Google Scholar wants people to use their product. To that end, they’ve introduced features like subject-specific citation leaderboards, citation panels on user profiles, journal rankings and a new Google Scholar Labs AI interface. However, Google has repeatedly demonstrated that they are not really interested in maintaining the integrity of their product,” he said.

Google did not respond to a request for comment.

Although some might regard the offence as harmless, academics should make more effort to curate their online profiles more closely given that “incorrectly assigned authorships have the potential to mislead people who may simply glance at the numbers rather than zoom in on the article titles”, said the professor who contacted THE.

“Maybe universities’ senior management should start instructing their academics to undertake regular housekeeping of their Google Scholar profiles,” they suggested.

jack.grove@timeshighereducation.com