Can AI Detectors Actually Tell If You Used ChatGPT? -- We Built One and Measured

Can AI Detectors Actually Tell If You Used ChatGPT? -- We Built One and Measured

We implemented an AI-text detector ourselves and measured when it works, when it accuses innocent people, and how it gets defeated -- detection and false-positive rates, in numbers.

A student flagged for "suspected AI writing" on an essay they wrote themselves. Companies screening applications with AI detectors. And, from the other side, the claim that "detectors are trivial to beat." Who's right?

Instead of arguing, we ran the experiment. We implemented AI-text detection under its most favorable possible conditions -- a vendor-planted watermark checked with the vendor's own key (the SynthID-Text family, which Claude adopted) -- on a local model, and measured exactly when detection works and when it doesn't. This post is the plain-language summary; the full experiment with code lives in the technical deep dive.

This is Part 2 of the "How AI Actually Works" series. Same rule as always: one thing you actually experience, one experiment we ran ourselves.

1. There are two kinds of detectors -- we tested the strong kind

First, a distinction. AI detectors come in two families.

• Classifier-based: they look at writing style and guess "does this feel AI-generated?" GPTZero and most tools used by schools and companies are in this family. Because it's a guess, they're known to misflag formal writing and non-native English speakers.
• Watermark-based: the AI company plants a statistical signature at generation time and checks it with its own secret key. This is closer to cryptographic verification than guessing -- it's effectively the upper bound of detection performance.

We tested the second, stronger kind. So read every number below as the best case for detectors. The classifier tools you actually encounter do worse.

2. When it works, it really works

We checked 50 AI-generated texts of ~200 tokens (about 150 words). With the right key, the detector caught all 50 (100%). The score distributions of AI text and non-AI text didn't overlap at all. Human-written paragraphs passed cleanly at 0%.

So "detectors are all garbage" is wrong. Given the right conditions -- long enough, unedited text -- detection is nearly perfect.

3. But on short text, it convicts the innocent

Same detector, shrinking the text length:

!Detection and false-positive rates by text length

Look at the right column. Point a detector at a sentence or two, and a third of human-written text gets flagged as AI. Statistical signatures need a sample to show up in, and short text has no sample. This isn't a quality problem -- it's a limit of principle that no detector escapes.

The practical conclusion: an AI verdict on short text -- an email, a comment, one paragraph of an application -- is not trustworthy, whatever tool produced it. Nobody should be penalized on that basis.

4. Light editing does not beat it

We also tested the folk claim that "changing a few words fools the detector." Swapping words in AI text at random:

Change one word in ten and everything still gets caught. To sneak half the texts through you need to replace 30% of the words -- and hand-mangling 60 words in a 200-word essay leaves you with garbage. Manual editing is not a realistic evasion.

5. But one "rewrite this" makes it vanish

The real hole is elsewhere: asking a different AI to rewrite the whole thing.

!Paraphrase attack results

One pass of "rewrite this, keep the meaning" through a small free model that runs on a laptop -- a few seconds -- drops detection from 100% to 8%. Same content, same structure, signature gone. Someone who wants to hide cannot realistically be stopped.

6. What this experiment does not tell you

• We measured the watermark family (the upper bound). Classifier tools (the GPTZero family) vary by product and are, in principle, worse.
• Operators can set more conservative or length-adjusted thresholds; our false-positive rates are what happens when a fixed threshold meets short text.
• English text only. Numbers may differ for other languages.

7. How to actually use this

Three lines:

1. Strong on long, unedited text. 150+ words of raw AI output gets caught almost surely. As a spam and content-farm filter, detection works.
2. Discard verdicts on short text. An "AI suspected" flag on a sentence or two is a verdict that also catches a third of human writing. Punishing a student or applicant on it means convicting one innocent in three.
3. Determined evasion wins. One rewrite defeats it -- which also means "passed the detector" is no proof a human wrote it.

A detector is not a judge of whether AI wrote something. It's a filter for unedited AI output distributed at scale. Ask more of it, and someone innocent pays.

*This post covered one question -- detection. For the whole picture of how these systems behave, there's a video course: How ChatGPT Actually Works: LLM Principles Without Code →*

Go deeper

• The full experiment (code and notebook): Reproducing Claude's Watermark — Part 2
• How the watermark works: Part 1
• Series Part 1: Why ChatGPT Gives You a Different Answer Every Time