If a chatbot can pass your assignment, you are grading the wrong thing

Two summers ago, I graded roughly 170 financial statement analysis projects in six weeks. Somewhere in the second stack, I realised that I was reading the same paper: the same interpretation of return on equity, near enough word for word the explanation on Investopedia, the same confident tone and – when I checked – the same citation to a real filing, correctly named, that said nothing of the kind. My students were not using artificial intelligence to do the analysis. They were using it to write the analysis, and my assignment could not tell the difference.

Like most academics, my first response to generative AI was to try to lock the classroom down: more proctoring, more detection software, sterner warnings on the syllabus. It did not work, and it was never going to. My students will use these tools every day of their careers. The question that finally helped was not how to catch AI but rather what I was asking students to produce. If a chatbot can generate a passing submission in 30 seconds, the problem is the assignment.

• Three shifts in course design to update pedagogies for the AI era
• Make AI literacy a foundational element of your course design
• Here’s how to show students what responsible AI use looks like

Here is what I changed in my financial statement analysis courses, and how you can adapt it, whatever you teach. The principle is simple: stop grading the answer and start grading the auditable trail that produces it. A large language model does retrieval superbly, because retrieval is what it is for. So, any task whose graded object is a clean final number or a tidy paragraph of conclusions measures nothing about the student’s thinking.

Three moves put the reasoning back at the centre.

Make source credibility a graded, non-negotiable standard

Every figure my students report must trace to a primary source – a 10-K filing, a 10-Q filing, an official filing on SEC EDGAR – and they must be able to point me to the exact page and line. This closes the two AI failure modes I see most. The first is easy to catch: a citation to a filing or URL that simply does not exist. The second is the one that matters – a real filing, correctly named, that does not say what the student says it says. The model summarised plausibly and nobody went back to the page. In my courses, a fabricated or AI-hallucinated source is not a small deduction. It is a breach of the standard, treated with the seriousness the profession gives it, because signing off on a number you cannot substantiate is how careers in finance end.

The general version of this approach is to require verifiable primary sourcing, check it, and make an unverifiable citation a real penalty rather than a shrug. Checking it is less work than it sounds, because the requirement is designed to make checking cheap: page and line, or it does not count. I sample rather than verify every figure, and I tell students that I sample. A credible chance of being checked, paired with a penalty they believe in, does more than exhaustive verification I could not sustain anyway.

Make the process the deliverable, not the output

In my courses, the submission is no longer a number or a conclusion. It is a live, auditable spreadsheet, and every cell that reports a figure must contain a formula that references a source cell – the actual line item from the actual filing – never a value entered by hand. When I mark, I click into the cells and read the formula bar. That sounds slower than scanning a final number, and per submission it is – but it fails faster: a hard-coded value or a broken reference is visible in seconds, and I stop reading there. What makes it tractable is that every student submits on the same template, so I am not learning a new layout 170 times. I check a fixed set of cells in every workbook rather than reading every formula in it. I marked roughly 170 submissions this way in a six-week summer term.

A days-sales-outstanding calculation that quietly multiplies by 90 instead of by the days in the period shows me exactly where a student’s understanding broke, and no polished narrative can hide it. A student who pastes in an AI-generated number has nothing behind the cell; the formula bar is empty, and that absence is louder than any plagiarism report.

Whatever your discipline, the equivalent is to grade the working rather than the endpoint: annotated drafts, tracked-change histories, code with its commit log, a marked-up set of sources. Assess the chain, not the destination.

Set problems that expose confident nonsense

Make judgement the assignment; it is the one thing that does not transfer to the machine. Ask a chatbot to evaluate and interrogate claims and you get confident, hedged mush.

So, I ask students to value a sportswear company through a stretch of negative free cash flow, where the standard textbook formula simply breaks and they have to reason their way to an alternative rather than plug and chug. I hand them a short-seller report and ask them to adjudicate its claims: which hold up, which are rhetorical, and what would a careful analyst conclude?

Outside finance, the move is the same: favour genuinely contested, ambiguous or context-specific prompts over anything with a findable right answer.

None of this requires exotic technology or a surveillance arms race. It does cost time: designing genuinely contested cases takes longer than recycling last year’s problem set. But it has quietly solved the AI problem in my classroom – not by banning the tool but by making it irrelevant to the thing I assess. My students still use AI. They simply cannot use it to skip the learning, because the learning is the only thing left on the page.

So, here is the test I would put to any colleague staring down the same anxiety. Open your next assignment and ask honestly: could a chatbot pass this? If the answer is “yes”, you do not need a better detector. You need a better assignment.

Lucas Long is an assistant professor of finance in the Lewis Bear Jr. College of Business at the University of West Florida.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.