AI can help with research, but humans must remain accountable

At a time of rapid technological advancement and widespread job losses around the university sector, academics could be forgiven for wondering whether their role in the research process might ultimately be made redundant by artificial intelligence.

Such AI systems – including but not limited to large language models (LLMs) – are certainly reshaping research practice at a breathtaking pace.

It is not that the underlying structure of research is changing. Whether in science, engineering, the social sciences or the humanities, this still typically involves six stages. These involve selecting research problems and questions; assessing the state of current knowledge; generating specific ideas and hypotheses; conducting the research (experiments, simulations, modelling, archival work, fieldwork or creative enquiry); analysing and interpreting the results; and, finally, communicating them.

But AI now influences every stage of this workflow. For instance, LLMs are highly effective at specific tasks such as literature review, idea generation and synthesis. Their speed and efficiency are transformative. However, here, as elsewhere, human domain expertise and oversight are essential to ensure quality, rigour and meaning.

LLMs can correctly identify highly cited papers and summarise domain positions. Yet influence in research is contested and shaped by disagreement. Not everyone agrees on what constitutes the most influential and important work. Scientific judgement cannot be reduced to metrics or consensus alone.

In the formulation of research questions, LLMs can quickly suggest avenues that sound credible and technically sophisticated. However, without both experimental and theoretical domain expertise, developed through study and research, they struggle to establish whether these problems are truly open, scientifically valuable, strategically important, or worthy of sustained investigation.

Equally, moving from plausible questions to new knowledge requires human-led critical thinking. Assumptions must be challenged. Contradictions must be explored. Risks must be taken and uncertainty embraced. These actions require responsibility, imagination and courage. AI systems can support this process but cannot replace humans’ intellectual responsibility for it.

Of course, some AI enthusiasts note that the pace of progress in AI is so rapid that any assertion of what AI cannot yet do has a very short shelf life. Yet as these systems become more powerful, the need for human responsibility only increases. Full automation of discovery may be possible in narrow cases, but it is neither realistic nor desirable as a general model. Capability without judgement undermines trust.

LLMs and humans interact most usefully as peer reviewers of each other’s contributions. Humans should use their expertise and judgement to assess AI outputs for usefulness, accuracy and bias. One useful piece of prompt engineering to learn in this regard is to require LLMs to expose their reasoning step-by-step, allowing researchers to inspect its logic and identify weaknesses.

LLMs, in turn, should be used to question humans’ assumptions, exposing flaws in explanation or design, and offering alternative perspectives. But in all cases, the role of the LLM is to stimulate critique, not to assert authority. The value of LLMs in research is not in the answers they give but in how they are used to support critical thought. “Recursive reflection” prompts, for instance, ask LLMs to refine ideas through multiple cycles of feedback, while “inversion” prompts ask them to list and then invert assumptions to stress-test a hypothesis.

Structured prompting frameworks such as RISEN (Role, Instruction, Steps, End, Narrowing) support systematic research interaction by embedding clarity, constraints and critical intent into the prompting process. Used in this way, prompt design becomes an extension of research rather than a shortcut.

The most profound practice shift that is occurring relates to PhD supervision. This is evolving from a two-way relationship between students and supervisors to a three-way partnership that also includes AI. The student brings curiosity, creativity and effort. The LLM contributes computational intelligence and knowledge access at scale. And the supervisors provide wisdom, values and judgement, ensuring intellectual growth, responsible use and ethical practice. Again, LLMs cannot replace supervision; they make good supervision more important than ever.

Nor can LLMs be authors on papers. Authorship remains inseparable from intellectual contribution, responsibility and accountability. Even when they meaningfully assist the research process, LLMs cannot be held accountable or answer for consequences.

As research automation increases, humans gain time to think more deeply about research questions, interdisciplinary integration, engagement with complex challenges, and economic and societal impact. The opportunity is clear: to push the boundaries of discovery by thinking harder, not just faster.

Of course, we are not deaf to the environmental concerns about AI use. But here again human direction and responsibility make a crucial difference. In many scientific applications, AI systems do not operate autonomously but are directed by researchers, who constrain the search space and prioritise promising hypotheses. This reduces the computational burden that would otherwise arise from exploring an enormous number of possibilities, thereby lowering energy consumption. At the same time, AI-assisted research can accelerate the discovery of new materials and technologies, such as more efficient batteries or energy systems, which may themselves reduce energy use.

Nor is climate change the only big societal challenge to which responsible and sustainable AI use might provide solutions by expanding analytical capacity and integrative thinking. Health and social resilience could also greatly benefit.

The main risk we face is not that AI becomes powerful or power-hungry. It is that humans disengage. If researchers remain curious and accountable, embedding AI use within rigorous peer review and transparent validation, then AI systems will remain what they ought to be: powerful tools that amplify human ingenuity rather than corrupt, diminish or overwhelm it.

Bashir M. Al‑Hashimi is vice-president (research and innovation) at King’s College London. Nick Jennings is vice-chancellor of Loughborough University. LLMs were used as tools to support drafting, refinement and critical exploration of ideas in this article. All interpretations, arguments and conclusions are the authors’ own.