Anthropic's "Watermark" Is One Line of Code
Hola — this is Liam, in for Bear. Anthropic says Claude's text carries a watermark. I cloned the reference implementation for this whole family of techniques. Can you walk me through what the mark actually is?
The Watermark Is One Line of Code
Hola — this is Liam, in for Bear. Anthropic says Claude's text carries a watermark. I cloned the reference implementation for this whole family of techniques. Can you walk me through what the mark actually is?
Topic: CLAUDE · CODE
Chapters
0:00 Introduction 0:34 WHAT 1:16 SHAPE 2:19 PATH 3:18 HISTORY 3:52 READ 7:00 CHANGE
YOUR TURN
@NikBearBrown
Sources and citations: see the reel's SOURCES.md in the open textbook.
▶ The full series playlist — Brutalist https://www.youtube.com/playlist?list=PLG9H-C6rp5RU
What this video covers
- 0:00 — Introduction
- 0:34 — WHAT
- 1:16 — SHAPE
- 2:19 — PATH
- 3:18 — HISTORY
- 3:52 — READ
- 7:00 — CHANGE
Transcript
A lightly formatted transcript of the video, with timestamps so you can jump to any section on YouTube.
Introduction
[0:00] Ola, this is Liam in for Bear. Anthropic says Claude's text carries a watermark. I cloned the reference implementation for this whole family of techniques. Can you walk me through what the mark actually is? The watermark is a statistical bias, not an inscription. At generation, half the vocabulary gets a two-logit boost at every token position. At detection, a Z-test asks whether too many boosted tokens appeared by chance. The algorithm is 282 lines. The surrounding apparatus is 344 000 lines of file. The detection returns a P value. A P value is not a verdict. The
WHAT
[0:34] Word watermark comes from images where there is a thing in the file you could point at. There is no such thing in the text. Nothing here is embedded. The repository jwkirchhoff/lm-watermarking, official implementation of the 2023 ICML paper by Kirchhoff and colleagues. 16 commits since the first push. Built on the Hugging Face Transformers logits processor abstraction, the hook that intercepts probability distributions before sampling. Logits processor matters. It doesn't reach inside the model. It sits between the model's output distribution and the sampling step, and it adjusts logits. The model's weights are unchanged. The watermark lives in the selection rule, which token the sampler picks, not in any parameter.
SHAPE
[1:16] Now, the shape of the code. Seven Python files at the root. demo_watermark.py is the CLI runner at 701 lines. watermark_processor.py is 282 lines, the mechanism. extended_watermark_processor.py is 625 lines, more on that one shortly. Then normalizers, alternative seeding schemes, the spaces app. Four subdirectories. watermark_reliability_release has 252 512 lines of file. experiment_pipelines, evaluation_json. hf_hub_space_demo has 44 191. homoglyph_data has 42 351 for Unicode lookup tables. The number is large. The claim is about files, not source. The ratio: 2,317 root source lines, 344,000 lines of file across the four subdirectories, but most of that is Unicode lookup tables and experiment JSON, not source. Watermark processor. Pi is 282 lines. That is 12% of the root and less than 1/10 of 1% of the tree. The idea is 282 lines. Everything around it is apparatus. Now, what calls what?
PATH
[2:20] The import graph app pi imports demo watermark. Demo watermark imports watermark processor, the mechanism. Watermark processor imports normalizers. Normalizers import homoglyphs. Four files in the full repo import watermark processor. The demo runner, the hugging face demo, and two files in the reliability release. Watermark processor pi 282 lines, four importers. Every call that matters routes through this file. If you want to understand what this algorithm does in code, that is the file. And then there is extended watermark processor pi. 625 lines. The readme calls it recommended, and not one file in this repository imports it. One more finding, three copies of watermark processor pi in the tree. Root 282 lines, hugging face demo copy 280 lines, 32 differing lines. Reliability release copy, 672 lines, 592 differing lines. Three divergent branches, none of them connected to each other.
HISTORY
[3:19] The get log. 16 commits across three years. 2023 12, the initial push, the paper release, the extended implementation, a Jonas japing documentation pass. 2024 2, 2025 2, the most recent in September, a hugging face spaces demo update. Every year has at least one commit. 16 commits is a paper artifact being maintained. This family of techniques is deployed at production scale. The reference implementation has 16 commits. That is not a project with dedicated resources. That is a researcher keeping the lights on.
READ
[3:52] Time to read the code. Five parameters, gamma zero, five half the vocabulary is green at each position. Delta two zero, the logic boost applied to green tokens. Hash key 50,485,863, the secret described in the code is just a large prime. Seeding scheme simple one, the RNG seeds on the single previous token only. Z threshold four zero, the detector's decision boundary. Three functions. First, how the key and the previous token seed a fresh random partition of the vocabulary at every position. Animate how the green list is built at each token. At every token position, the mechanism draws a fresh random permutation of the full vocabulary seeded by the secret key and the previous token. The first half of that permutation is the green list. It changes completely at the next position. It is never stored. There's no green list file anywhere in this repository. The seeding and partitioning code, line 63, the previous token times hash key seeds the RNG. That is the entire secret. Line 74, torch ran perm shuffles the
Full vocabulary using that RNG. Line 76, the first gamma fraction of the shuffle is the green list. Range shortened to L6280, L61, the assert overflows the frame at the type floor C build log MD. The green list is not stored anywhere. It's regenerated from scratch at every token from the secret key times the previous token. Delete the key and it is unrecoverable. Now show the bias high entropy versus low entropy. When the model is uncertain, high entropy multiple near equal options adding two zero to the green logits shifts which token is sampled, choice flips. When the model is confident, Barack followed by Obama, the top token wins regardless of the green boost. The watermark is invisible at low entropy. It only bites when there's a real choice. Line 95, scores on the green list get plus green list bias, which is delta, which is two zero. That is the entire marking mechanism. Everything else in call is batch handling, RNG setup, and the calls to the seeding and masking functions. Scores green
List mask equals scores green list mask plus green list bias. That is the watermark, one line add two to half the logits. Now the test, 200 tokens versus 40. Detection is a hypothesis test. Under the null, no watermark approximately gamma times T tokens fall green by chance. The Z statistic measures how many standard deviations above chance the observed count sits. At 200 tokens, the test has power. A watermark text clears the Z threshold of 40. At 40 tokens, the distribution is wide and the same observed count does not clear the threshold. The test returns a P value, not a ruling. Lines 153 163, the Z score observed green count minus expected divided by the binomial standard deviation. The P value scipy stats norm sf, the survival function one-tailed area above Z. This function returns a float. The threshold comparison happens in the caller. Detection is scipy stats norm sfz. It returns a P value. A P value is not a verdict, and asking to see the loadedness of a single die roll.
CHANGE
[7:00] One commit. Commit bbe1fb8, August 2023. Message documentation updates the stat for watermark processor pi. 17 lines added, 15 removed. Most of it is genuine cosmetics, blank lines, pp8 comment spacing, but one line is not documentation. Buried in the watermark detector in its signature, the default value of ignore repeated bigrams changed from false to true. That is not documentation. 17 lines added, 15 removed. Message documentation updates. One of them flipped a detector default from false to true. The key line is where the error hides. Ignore repeated bigrams changes how observed count is computed, whether repeated token bigrams get counted once or every time they occur. That changes the Z-score. That changes the P-value. That changes the detection verdict for any text that sits near Z threshold of 4.0. A commit label documentation updates change the detection statistic. The verdict, the
Algorithm is honest. The mechanism is 282 lines, and it does exactly what the paper says. But, the mismatch between what the readme recommends and what the tree imports is a real gap in the commit that said documentation updates change the scoring rule. If you audit a text and the detector says no, you are reading a P-value above 4.0, not a ruling. The watermark is statistical, so is the doubt. Your turn. Take this into your own session. Clone jwkirchhoff/lm-watermarking open watermark processor. Pi and ask Claude what happens to detection power when you raise delta from 2.0 to 5.0 and what the model quality tradeoff looks like at high delta. Then, ask what Z threshold controls and what a false positive looks like at 200 tokens with no watermark present. Liam in for bear. The watermark is one line of code.
