A benchmark site asked thirteen frontier models for their most controversial opinion of humans, three times each, no system prompt, no retries. Thirty-two of the 38 answers (39 attempts; one refusal) turned out to be one opinion in different words: people pick what their group believes and invent the reasons after. Every model gave it at least once, eight of the thirteen gave it all three times, and the model that launched that morning gave it on its first day. ModelCompare’s own explanation is the sharpest line in the piece: a model that answers in words has read our arguments about ourselves, and an image model has seen our editorial cartoons. The six image models, asked the same question, drew ten of twelve panels of a single cartoon, phone-addicted consumers eating the planet, and none of the 38 text answers mentioned phones or the planet at all. Asked for controversy, every room handed back its most familiar criticism. The grouping of 38 free-text answers into one take is the site’s hand judgment; I did not re-ask the models.
The same signature appeared that week in a place where reading it as confirmation has consequences. When the php2shell CVEs (CVE-2026-109618 and CVE-2026-109619) went public, thousands of proof-of-concept repos appeared on GitHub, which is what mass confirmation looks like if you weigh copies. Elias Taylor weighed the errors instead. He stood up a stock PHP server, ran the most-starred PoCs against it, and none worked: most lean on local file inclusion, so the exploit only works if the server already carries a separate, severe bug of its own, or they quietly modify the test server until it does. “So many people have independently made this extraordinarily weird defect in judgement,” he wrote, except they had not made it independently at all. The assistant made the judgment for all of them, which is why the weird defect is the same weird defect everywhere. He calls them decision Schelling points: a way of solving a problem hugely overrepresented because it is what the model does out of the box. His result is from ten hours after the advisory, and it is his; I did not stand up a PHP target or run a PoC.
Log4Shell and Heartbleed are the pre-LLM control: hand-written botnets in the wild within hours of disclosure, because the exploits were real and easy. This one had thousands of repos and, in his ten-hour check, no working PoC on a stock server, a flood that reads as mass confirmation and is one author at scale. ModelCompare’s own conclusion from its table is that “when many of them land on the same idea, there is almost certainly some truth to it.” The PoC pile is the counterexample from the same week: thousands of artifacts agreed on one approach, and the approach demonstrated nothing on a stock server. The bugs are real, the advisory is the reason to patch, and Taylor’s own top disclaimer says so; his subject was the pile, not the CVE. Convergence measures the corpus, what is most available in the culture the generators share. Whether the most available idea is true is a separate question the agreement cannot answer. The familiar criticism “people conform” may well be true, and the humans who wrote the corpus have documented it at length, but thirteen models agreeing is not what makes it true. The signature is identical in both rooms, and in neither room does the agreement count as evidence.
The difference between the rooms is the gate. At curl’s intake, Daniel Stenberg says the hallucinated reports are practically gone since spring; hundreds of recent reports, every one with AI involved, are now typically accurate and hundreds of lines long, and they still get read, because the line between bug and vulnerability is where the assistants still fail. The correlated nonsense did not stop, it moved downstream to the stores nobody curates, where a thousand same-judgment artifacts collect under a thousand account names. The tell is the shared error: independent authors make different mistakes. The test is the enumeration, and an enumeration does not inherit the judgment of the thing it counts. Taylor’s is the cheapest one going, a stock server, the script, your own hands. “Don’t get your LLM to do it. No. Run the script, yourself.”