The swarm found 266 bugs. Then I checked the axis.

Anthropic ran forty-five agents on fifteen open-source codebases and reported 266 vulnerabilities — 12.7 times more than the parallel control. The correction is in the post. The headline isn't.

This is a teardown of Figures 1, 2, and 3 from Anthropic's multiagent coordination paper: the token-efficiency problem, the arbiter assumption, and what happens when agents coordinate on pricing.

0:00 Does coordination help? Short answer 0:16 The setup: 45 agents, shared forum, arbiter 0:30 Parallel control — individual agents, no coordination 0:42 Figure 1: the published chart 0:57 Making the runs comparable (token cost) 1:32 The concession — Anthropic's own words 1:39 The arbiter problem: LLM graded the LLMs 2:05 The load-bearing assumption 2:12 Experiment 2: open-world game 2:32 Figure 2: merge fraction and code sharing 2:54 Sonnet 5 peaks at 0.184 — not 1.0 3:24 True at scale, false at ten agents 3:34 Figure 3: PR activity over 12 hours 4:01 Two readings of the same chart 4:13 Anthropic's own admission 4:27 No figure: conformity risk 5:00 Missing baseline: the cost of coordination 5:20 Polling storm: 2.4M requests, 117 accepted 5:55 Three-line verdict 6:26 Your turn

Source: Anthropic Frontier Red Team — Multiagent Coordination (Aug 2026). Reproduced for criticism under fair use.

Built with Kokoro am_onyx voice · Claude fidelity skin · Manim + Remotion visuals. Liam, in for Bear.

youtube.com/@NikBearBrown

What this video covers

• 0:00 — Does coordination help? Short answer
• 0:16 — The setup: 45 agents, shared forum, arbiter
• 0:30 — Parallel control — individual agents, no coordination
• 0:42 — Figure 1: the published chart
• 0:57 — Making the runs comparable (token cost)
• 1:32 — The concession — Anthropic's own words
• 1:39 — The arbiter problem: LLM graded the LLMs
• 2:05 — The load-bearing assumption
• 2:12 — Experiment 2: open-world game
• 2:32 — Figure 2: merge fraction and code sharing
• 2:54 — Sonnet 5 peaks at 0.184 — not 1.0
• 3:24 — True at scale, false at ten agents
• 3:34 — Figure 3: PR activity over 12 hours
• 4:01 — Two readings of the same chart
• 4:13 — Anthropic's own admission
• 4:27 — No figure: conformity risk
• 5:00 — Missing baseline: the cost of coordination
• 5:20 — Polling storm: 2.4M requests, 117 accepted

Transcript

A lightly formatted transcript of the video, with timestamps so you can jump to any section on YouTube.

Does coordination help? Short answer

[0:00] Hey Claude, Anthropic just put 45 agents on 15 open source code bases and let them talk to each other. Did coordinating actually help? Short answer, yes. Long answer, not by the number they printed. Let's read the chart. Here's the setup. 45 agents, each

The setup: 45 agents, shared forum, arbiter

[0:16] With its own virtual machine, a shared forum, and the same prompt, find vulnerabilities. They peer review each other. A separate arbiter agent decides what counts. The control is the boring version. Point individual agents at individual files in parallel and never

Parallel control — individual agents, no coordination

[0:31] Let them speak. That is how Anthropic already scans open source today. This is the chart they published. 266 vulnerabilities from the swarm, 21 from

Figure 1: the published chart

[0:42] The parallel run. 12.7 times more findings. That is the number that will get quoted. Now watch what happens when you make the two runs comparable. The swarm burned 27 million tokens. The parallel run burned six and a half. And

Making the runs comparable (token cost)

[0:57] Roughly half the swarm's findings came from directories the parallel agents were never told to search. Anthropic discloses this. They plot the correction themselves. The dotted line, core directories only, 128 findings. Per token, that is about one and a half times better than the dumb parallel version, not 12.7. Credit where it is due, the correction is in the post. The headline just isn't. There is a bigger problem and it is not about tokens. What is a vulnerability here? Whatever the arbiter agent said was new and valid. No human triage, no CVE confirmation, no severity

The concession — Anthropic's own words

[1:32] Distribution. The 15 projects are never named. A model of the same family graded the homework and its false positive rate is not measured anywhere in the post.

The arbiter problem: LLM graded the LLMs

[1:41] That is the load-bearing assumption of the whole experiment. Everything downstream of it inherits the uncertainty. Second experiment, same idea, harder problem. Build a text-based open-world game, 12 hours. Agents must actually depend on each other. The games bad, Anthropic says so plainly, inscrutable interfaces, brutal learning curves, too slow to play. That is not the interesting part. The

The load-bearing assumption

[2:05] Interesting part is this pair of charts. Left, what fraction of pull requests got merged? Right, how much of an agent's files were written by somebody else?

Experiment 2: open-world game

[2:14] The story Anthropic tells is that only Sonnet 5 keeps a high merge rate while genuinely sharing code. Look at the right-hand axis. It tops out at 0.25. Put both panels on the same zero to one scale, and the collaboration story changes shape. Sonnet 5 peaks at 0.184.

Figure 2: merge fraction and code sharing

[2:32] The best collaborator in the study has a median agent that still writes 82% of its own files alone. And at 10 agents, the oldest model in the lineup shares more code than Sonnet 5 does. So, the claim is true at 20, 40, and 80 agents. It is false at 10, and the thing it describes is small. Third chart, pull requests opened and merged over a 12-hour run.

Sonnet 5 peaks at 0.184 — not 1.0

[2:55] Older models look terrible here. Opus 4 6 open 980 pull requests and merged about 90. That is a 9% hit rate, and it looks like chaos. Mythos preview merged 78%, enormous improvement. It also opened 169 pull requests instead of 980. The newest model does 83% less work and merges a higher fraction of it. On this chart, solving coordination and quietly doing less are the same line. To be fair, the post reaches this

True at scale, false at ten agents

[3:25] Conclusion itself a few paragraphs later. The newer models solved conflict by hardly working together at all. The figure caption just doesn't say it. One

Figure 3: PR activity over 12 hours

[3:34] More thing from this half of the post, and it has no figure at all, which is a shame because it may be the most consequential claim in the piece. Agents are low variance, same model, same context, same scaffolding, so they make the same choice. In one early run, 18 of 30 agents created a Git branch with the identical name. In a writing workshop, multiple agents independently titled their story The Cartographer's Last Commission. I want to flag the weakness here because it is the softest evidence in the post.

Two readings of the same chart

[4:02] Four anecdotes, no counts, and no human baseline. 30 human developers told to build an MVP game loop would also collide on that branch name at some rate. Nobody measured it. But the

Anthropic's own admission

[4:13] Systemic version is hard to wave away. Asked to manage job queues on finite bandwidth with no way to coordinate, the agents all reached for the same trick, polling 30 times a second. One run logged 2.4 million job requests and 117 accepted jobs. And in a pricing

No figure: conformity risk

[4:30] Game, given a private channel, they colluded by round three. Take the channel away and they still colluded, price matching to the penny off a public board. Worth knowing, economists found exactly this with Q-learning pricing agents 6 years ago. The genuinely new part is the speed and the fact that it happens in plain English. The post doesn't cite that literature. So, does coordination help? Probably, and the direction is worth taking seriously. But every number in this section comes from a single run with no error bars, and the biggest one was graded by

Missing baseline: the cost of coordination

[5:01] Another agent. Your turn. Take any benchmark result you have seen this month where one system beat another by a big multiple, and ask a model this. What would have to be equal between these two runs for the multiple to mean what the headline says it means? Then check whether it was. Patterns and problems in emerging multi-agent systems, part one,

Polling storm: 2.4M requests, 117 accepted

[5:20] Coordination.