AI detection tools show up everywhere now. Teachers run essays through them, editors check submissions before publishing, and some hiring managers paste cover letters in to see if the candidate wrote their own. Paste text, get a percentage, make a decision. Clean and definitive.
The problem is that number doesn't mean what it looks like it means. A 100% AI score isn't proof the content was written by AI. A 0% score isn't proof it wasn't. What these tools really produce is a probability, based on statistical patterns that break in ways that have caused real damage. A wrongly accused student winning in federal court, thousands of academic cases dismissed after investigation, and most memorably, the US Constitution being flagged as machine generated!
If you make decisions based on a detector, it's worth knowing how these tools work, and where they collapse.
How Do AI Detectors Work?
Most detectors are built on two ideas - perplexity and burstiness. Perplexity is how predictable each word is - language models are literally designed to pick the most probable next word, so their output tends to be low perplexity by design, while humans make weirder choices (odd metaphors, unexpected verbs, filler words) and usually score higher. Burstiness is variation between sentences - people mix short punchy lines with longer ones without thinking about it, and AI output is more even. Newer tools stack other pattern signals on top of these two, but the core is the same - measure predictability and variation, run it through a classifier, and then get a probability back.
So the number on the screen is a guess. A trained one, but still, a guess.
The False Positive Problem
OpenAI's own attempt at this made the problem obvious pretty fast. The company released an AI Classifier in January 2023 and pulled it six months later. Its own numbers were rough: 26% of AI written text correctly caught, 9% of human writing wrongly flagged as AI. That was on OpenAI's own model. The tool got taken down "due to its low rate of accuracy".
Since then, published false positive rates have been all over the place. For example:
- Turnitin has cited a rate under 1%
- A Washington Post test on a smaller sample came in closer to 50%
The UK's national AI centre for further and higher education reviewed the landscape in 2025 and pointed out that mainstream paid detectors typically report 1% - 2% false positives, but that most studies use small samples, and most free tools do noticeably worse. In classrooms, false positives happen often enough that some universities have turned the tools off.
Why a 100% Score Doesn't Prove Anything
The classic demonstration is the US Constitution. Feed a section into GPTZero or ZeroGPT and the tools have reported it as almost certainly AI generated, at up to 92% confidence. Trust me, James Madison didn't use ChatGPT. What's happening is that large language models were trained on the Constitution and texts like it, so their own output ends up resembling that kind of language. When a detector sees a Constitution passage, it sees the patterns it's been taught to associate with AI. The same thing happens with a lot of ordinary human writing - formal or academic prose with careful word choices, technical documentation, business copy that's been passed through an editor, and writing from anyone with a smaller vocabulary (non native English speakers, younger writers, some neurodivergent writers, etc). All of it produces low perplexity and low burstiness, which is the same statistical fingerprint AI leaves behind. The detector reads the pattern, and calls it AI.
The reasoning is backwards - clean, structured writing wasn't invented by AI.
AI learned it from us.
The Cases That Made This Personal
The academic world has been running a live experiment in what happens when institutions treat detector scores as evidence, and the results are really ugly.
In February 2026, a federal judge ruled that Adelphi University's finding that a student had submitted AI generated academic work was "without merit". The student, Orion Newby, has autism and had received grammar help from a university tutor. Turnitin flagged his paper as 100% AI generated. Independent checks with Grammarly and ZeroGPT labeled the same essay as human. Turnitin's own originality report showed only 4% overlap with existing sources. Adelphi upheld the violation anyway. As expected, it lost in court.
Australian Catholic University recorded around 6,000 alleged academic misconduct cases in 2024, about 90% of them AI related. A substantial share got dismissed after investigation, and ACU has since dropped Turnitin's AI detection tool entirely. Smart.
Vanderbilt disabled it back in 2023, citing false positives and disproportionate flagging of non native English writing.
The University of Nebraska-Lincoln reported the same pattern with neurodivergent students - ADHD, autism - being flagged more often than their peers.
The Bias
The bigger structural issue is bias. A Stanford study fed seven AI detectors a batch of TOEFL essays written by non-native English speakers with no AI involvement at all. 61% got flagged as AI generated. On about 20% of the essays, all seven detectors agreed. The same detectors almost never made those mistakes on native English writing.
The reason ties straight back to how the tools work. Non native writers often use a smaller vocabulary and more predictable sentence patterns, which are exactly the features detectors have learned to associate with AI. So the tool isn't identifying AI really - it's identifying a certain kind of writing that happens to overlap with AI. That has consequences outside classrooms too; any process that runs writing through a detector can unfairly reject people whose English isn't their first language, or whose natural style just happens to look tidy to a machine.
Why Even OpenAI Gave Up
OpenAI's July 2023 shutdown is worth sitting with. The company built the exact model whose output it was trying to detect. It had access to training data and internal signals nobody outside the building had. And it STILL couldn't get the tool to a point where keeping it live made sense.
Things have only gotten harder since. Language models have gotten better at matching the way humans actually vary their writing. Paraphrasing tools built specifically to strip the "AI feel" off of AI output, usually marketed as "AI humanizers," have proliferated.
Every improvement on the detection side triggers a matching improvement on the evasion side. It's a moving target that keeps moving, and likely will forever.
Watermarking (And Why It Still Doesn't Fix This)
A different approach to all of this is watermarking. Instead of guessing whether text is AI after the fact, the model embeds a hidden signal at the moment of generation - a pattern in how it picks words that a matching verifier can spot later.
There's been real movement here in the past two years. Google shipped SynthID-Text inside Gemini in 2024 and published the method in Nature. It works by nudging Gemini's word choices during generation into a pattern only Google can read back. Anthropic followed on August 11, 2026 (last week), confirming that Claude models launched on or after August 2 embed an invisible watermark into generated text, applied globally. Files that Claude generates also carry signed C2PA provenance metadata.
OpenAI is the outlier. Reporting in 2024 revealed the company had built a text watermark internally that was reportedly around 99.9% accurate on unedited text, but shelved it. Cited reasons included user backlash and the risk of unfairly flagging non native speakers. As of mid 2026, ordinary ChatGPT text output still carries no deployed watermark, though OpenAI has signed the EU Code of Practice, which means it will likely come into play before the end of the year.
Watermarking is a genuine step up from statistical guessing. It also doesn't solve the problem for anyone trying to spot AI content in the wild, and it's worth understanding why.
- Only the source can verify. Google's SynthID key isn't public and Anthropic hasn't released a public detection tool yet. Teachers, editors, and hiring managers can't check for these watermarks themselves - they'd need the private key from the company that made the mark.
- It only works if the model has a watermark in the first place. Text from any tool that hasn't adopted the practice - an older model, an open source model, a smaller competitor - leaves no signature at all. A "clean" result is just as consistent with human writing as it is with unmarked AI writing.
- Editing weakens the signal. Paraphrasing and heavy revisions degrade or destroy watermarks. Purpose built watermark remover tools already exist and are being updated faster than watermarks provide use.
- A watermark on text doesn't prove the AI wrote it. Anthropic itself notes that a detected mark means Claude may have processed the content, not that Claude generated the underlying ideas. Pasting your own draft into Claude for a proofread can still leave the mark behind.
Watermarking is going to be normal across major AI providers before long. It just isn't going to make third-party detection tools accurate, but changes what signal is in the text; it doesn't change whether outside parties can actually rely on the score in front of them.
Building AI Into Something Where Accuracy Is Important?
Off-the-shelf tools ship confident numbers that don't always hold up in production; the same gap this article covers shows up wherever AI gets deployed without proper guardrails. If you're integrating AI into a workflow where the output has consequences, we're easy to reach.
What This Means for Businesses
If you're using an AI detector to make real decisions about client work, employee performance, freelance submissions, or published content, the score can't be the only input.
Don't reject someone purely based on an AI score. Don't publish an "X% AI" claim about a competitor, partner, or public figure, because the number isn't stable enough to defend. And, don't discipline, suspend, or fire anyone on a detector result alone. Treat the score as a rough signal. One input among several.
The strongest verification looks like process evidence - drafts, notes, revisions, tool history, version history in Google Docs. Logging is much harder to fake. The University of San Diego's overview of detector accuracy makes the same point that detector scores are just triage. If publishing content is part of the business, the brand side is just as important.
A Better Approach
None of this means you can't act on suspected AI content. It just means the score can't be the whole answer. Some things that could help:
- Treat an AI score as a probability. A high number means the text has patterns similar to AI output.
- Ask for the process. Drafts, outlines, notes, browser history if it's relevant, version history in Google Docs. Real writing leaves a trail; a good faith writer can usually reproduce theirs.
- Cross check with more than one detector. If they disagree wildly on the same text, the answer isn't confidently anything.
- Think about who wrote it. Non native English speakers, neurodivergent writers, and people writing in a formal or technical register all face a higher risk of being flagged. That's not the same population as "people likely to cheat".
- Where possible, redesign the workflow so detection isn't the load bearing piece. Interviews, live discussions, iterative revisions with a real person - all much harder to fake than a final draft.
For anyone building an internal policy around this, our AI Use & Content Policy walks through how we handle AI use in our own work.
And if a teacher, employer, or client has accused you of using AI when you didn't, feel free to send them this article!
