Retaining human oversight of AI marking ‘complex’, trial finds

Ensuring humans retain oversight of marking done by artificial intelligence is “more complex than one might initially think” and may not save academics any time in the long run, according to the initial results of a major UK trial.

Select universities and colleges in the UK have been experimenting with ways to introduce AI into their assessment processes with support of sector body Jisc.

Findings published on 21 May identified a “tension” between the AI marking tools and maintaining academic oversight.

While the trial did not enforce many restrictions on universities’ participation, they were required to ensure that “wherever AI was used to mark and give feedback on students’ work, a human should retain oversight of the process”.

But the trial found that “keeping the human in the loop is more complex than one might initially think” and may not deliver time savings to the marking process. 

It raised concerns that academics could be “influenced by AI feedback, rather than reviewing it critically”, and academic oversight of AI marking could be “reduced to simply scanning one’s eyes over the student’s work and the AI feedback before clicking ‘next’”.

A blog of the findings says: “At first glance, this problem seems to be driven by a lack of diligence, but further insights revealed a more interesting logic. As per the objectives of the pilot, the motivation for using AI in this context is to reduce workload. 

“As such, there is an expectation that the AI’s outputs can be trusted and delegated to some extent. If the human in the loop is putting significant effort into reviewing the student’s work and the AI’s feedback, then what is the purpose of using AI at all?”

Tom Moule, senior AI specialist and product lead for artificial intelligence at Jisc, told Times Higher Education that rather than offering time savings, AI may allow academics to offer “richer”, “more detailed” and “more timely” feedback.

Viewed through this lens, AI allows academics to “do more with the same”, he said, offering more useful feedback with no extra effort.

“The way to resolve that tension is to have a clear separation of what the role of the human and the AI is. In doing that you can verify that it is the human taking the dominant role and the role that requires the highest level of expertise, and the AI doing the role that requires perhaps more routine, but time-consuming tasks,” he said.

For example, academics could review and make notes on the student’s work before seeing any AI feedback, putting them in a “stronger position to review the AI’s first draft of feedback discerningly”. 

“In terms of time savings, the key here is for the human input to be focused on the overall effectiveness of the student’s work and some of its standout aspects. The AI, meanwhile, plays the role of analysing fulfilment of the marking criteria and writing up the feedback in rich detail,” the blog says.

This should be built into the foundations of any marking tools, it argues. “Ultimately, the pilot is reinforcing an important lesson: human oversight works best when it is designed intentionally into the workflow. When educators are given a clear role in shaping, verifying and refining AI-generated feedback, the technology becomes more of a collaborator rather than a substitute.”

The blog also stressed that AI feedback is most effective in formative assessment, and such tools are best suited to “delivering rich, actionable feedback to students in a timely manner”, rather than working on summative assessments, where “the mark is the primary concern”.

Some institutions raised concerns that the tools could “produce slightly different outputs” if put through multiple times, but the paper argues “variation is not unique to AI; different educators can also arrive at different judgements”.

“In many institutions, existing moderation processes already exist to smooth out such differences,” it says.

juliette.rowsell@timeshighereducation.com