At the start of August, Anthropic announced that for new models of Claude, it would be adding machine-readable “watermarks” as a way of complying with the EU AI Act’s Code of Practice on AI transparency. Anthropic said “When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it.”
The sighs of relief across Higher Education were audible. This is familiar territory. Where there is plagiarism, let us bring a plagiarism detector. Where there is AI, let us bring an AI detector. And where there is a secret AI-revealing watermark, let us bring a watermark detector. Problem solved.
Except that it isn’t.
The existing crop of “plagiarism detectors” are not always very accurate at detecting plagiarism. This is because they are actually better positioned as “similarity detectors” which scour the online world matching word patterns and phrases. A finding of plagiarism is a human decision and the patterns from the detectors are background information, but that is all.
Although they are not yet widely available, why would we expect a watermark detector to be any different? As universities begin deciding whether and how watermark detection should feature in academic integrity policies, however, there is a danger that a transparency mechanism designed to identify AI-generated text becomes mistaken for evidence of academic misconduct.
Consider the three students discussed below. All have used GenAI as part of the process of constructing their assignments, but in different ways and with different outcomes.
Student 1 reads the assessment brief thoroughly, and uses institutionally-approved technologies to find appropriate sources. These are downloaded, read, annotated and considered, and then the student sits down to write their 2,000-word essay, adding in fully-referenced quotes from the sources they have found, and adding their own critical analysis. They are concerned about their poor grammar, however, as this has been highlighted in previous feedback (which they read and acted upon). They upload their completed essay to their GenAI of choice, and ask it to rectify any grammatical errors. Once this is done, they copy the revised text back into their assignment and submit.
Student 2 is rather less conscientious. They have not made time to do much preparation for their assessment, which they have realised is due tonight. Rather than wasting time reading the brief and the guidance, they upload it to their favourite GenAI and ask it to write the 2,000 words for them. They then remember something about a session earlier in the academic year on AI use, so they find and upload their institutional guidance on AI as well. They also make sure to ask the GenAI tool to reference properly. They then copy and paste the text verbatim and submit.
Student 3 also has a busy extracurricular life, but remembers the assessment deadline a few days earlier. They also don’t really have the inclination to read all of the material, so they get GenAI to write the assignment for them. One of their friends has mentioned in the café that there was a rumour that AI detectors were being used, and Student 3 is quite keen not to get caught out. They spend some time rewriting it: changing sentences, swapping words and paraphrasing passages until it sounds sufficiently different from the original. They also spend a little time double-checking that none of the sources have been hallucinated, though they do not have time to actually read them. They submit their finished assignment.
To summarise the actions of the three students:
Student 1
- Intellectual Effort: Student
- AI role: Grammar / polish
- Likely watermark: Possibly retained
- Integrity concerns: Low
Student 2
- Intellectual Effort: AI
- AI role: Writes assignment
- Likely watermark: Likely retained
- Integrity concerns: High
Student 3
- Intellectual Effort: AI
- AI role: Writes assignment, then student rewrites
- Likely watermark: Possibly removed
- Integrity concerns: High
All three students are probably recognisable as archetypes, and they are deliberately simplified. However, this simplification helps to expose the problem, which is one which Anthropic itself identifies.
There are potentially two main types of watermark which can be embedded into AI-generated text. The easiest to rinse away is the metadata, hidden-character, or hidden formatting type of watermark. A lot of this can be removed just by copying the output and “pasting as plain text” into a clean document. The other, which is a more stubborn stain, is a statistical or token watermark. This works by subtly biasing the model towards particular word or token choices, creating a statistical pattern that can later be detected. Pasting as plain text will not remove this, as it is embedded in the patterns of the text.
Student 1 is diligent, careful and has followed the rules and guidance on AI. They completed the intellectual work in creating their assignment, but if there is a statistical/token watermark on their grammatically altered text, then it may be picked up by a detector. Student 1 is likely to have to explain the rationale for this in a hearing.
Student 2 is neither diligent nor careful, and expended no intellectual effort in the creation of an assignment. The fact that they are likely to get caught is not a problem.
Student 3 is trickier. They did not undertake the intellectual work to create an assignment, and they deliberately and knowingly spent time changing their work sufficiently to potentially evade a detector. Anthropic itself acknowledges that heavily edited or paraphrased content “may not carry a detectable mark.” As far as the submitted work is concerned, it may not be flagged as suspicious. Student 3 is a successful AI launderer.
It is largely accepted that “AI detectors” (as opposed to “watermark detectors”) have a number of issues. They can give false positives (Weber-Wulff, et al., 2023) and may disadvantage particular groups of students, including those who are writing in a second language (Liang, et al 2023).
Similarity detectors, as mentioned earlier, are quite good at spotting where a series of words has appeared in an online published source and, with proper configuration, can be set to exclude text which is in quotation marks. Student 3 might be tempted to employ their own hidden watermark to evade this – adding quotation marks in white coloured font at the start and end of the assessment, so the tool treats the entire piece as a direct quote.
Watermark detectors will perhaps have some differences. Since they are looking for an embedded signal in the text, rather than trying to assess whether a piece of prose “looks” AI-generated, then the issue of bias against students writing in a second language may be reduced. Nonetheless, the examples above demonstrate the potential for an accurate detection to produce a misleading inference (Student 1), and the relative ease of likely evasion (Student 3).
None of these tools therefore is able on their own to answer the key questions which need resolving as part of the academic integrity process. A similarity score does not prove plagiarism; an AI detector score does not prove unauthorised AI use; and a watermark would not prove academic misconduct. At best, each provides some evidence about how a piece of text may have been produced.
The distinction which matters here is that between provenance and misconduct. A watermark may tell us something about where particular words came from, and who “wrote” them, but it cannot tell us who did the intellectual work behind them, whether the use of GenAI was permitted, whether it was appropriately disclosed, or whether the student has misrepresented someone (or something) else’s work as their own. Those are the questions an academic integrity process actually has to answer.
Treating a watermark as anything more than one piece of contextual evidence risks producing precisely the wrong incentives. Student 1 has little reason to conceal their use of GenAI, and may therefore leave the watermark intact. Student 3 has every reason to remove it, and has spent not inconsiderable time and effort devising a strategy to do so. The more heavily institutions rely on detection, the more we risk teaching students that the important distinction is not between appropriate and inappropriate AI use, but between detectable and undetectable AI use. Laundering, rather than creating, becomes the goal.
There is also a risk that the widespread use of tools such as this could allow universities to postpone a much more difficult conversation. If watermarking appears to offer a reliable way of identifying AI-generated text, there will inevitably be a temptation to treat it as a technological fix for an assessment problem. We have seen versions of this before. Similarity detection did not remove the need to teach students about attribution and plagiarism, and AI detection has not removed the need to rethink assessment in the age of GenAI. Watermarking will not remove the need to decide what kinds of AI use we actually value, tolerate or prohibit.
Universities should therefore be cautious about incorporating watermark detection into academic integrity procedures. It may be useful as a prompt for further enquiry, particularly when combined with other evidence, but it should not become a proxy for authorship or misconduct. More useful approaches are likely to include clearer declarations of permitted AI use, greater attention to the process by which work is produced, assessment designs which make intellectual contribution visible.
Alongside this, and probably more important than everything else combined, is the need to educate students (and staff) about which uses of AI are ethically, procedurally and contextually appropriate, which uses are not, and crucially, the rationale behind it.
