Who Can Tell?

Issue 449 · Published September 24, 2026

Digitally Literate is a weekly letter about the forces shaping how we read, write, learn, and live in digital spaces. Subscribe to get each issue by email.

Last week, we talked about the growing calls to slow AI down, which is a polite way of saying we spent a lot of time on whether AI will kill us all. I also spoke with Live 5 News about why these risks no longer feel theoretical.

This week, I kept running into a different question from three directions. Not whether AI is dangerous, but whether anyone is in a position to know. The people paid to test these systems are burning out. The best evidence in a dispute between an American AI company and seven Chinese labs comes from one of the parties. And at the UN, the people building AI asked the world for help watching it, the day after the U.S. president told the General Assembly he would reject any globalist scheme to govern it.

As always, my email is hello@wiobyrne.com. Let's get into it.

The evaluators are running out

The Financial Times reported this week that the people responsible for checking this technology are breaking down. Staff at the UK's AI Security Institute, the closest thing there is to independent testing of frontier models, have been signed off work with stress and put into counseling. The pressure has landed hardest on the teams testing whether models can find unknown software vulnerabilities or help create biological threats.

More than a dozen senior researchers have left OpenAI, Anthropic, and DeepMind in two years over safety concerns and the pace of development. Jacob Coxon, whose resignation was the subject of last week's issue, appears in the piece as one name among several.

Anxiety in the tech industry isn't new. What's different now is that it's the people whose job is to tell the rest of us whether these systems are safe and whether the people needed to make these decisions are still in the room. It appears we're slowly getting the answer. In May, AISI merged its societal resilience team into another unit, reducing the team from about 15 researchers to 3. The head of that team resigned in July.

We've watched this pattern hollow out other fields. Is this what it looks like in AI governance? Perhaps the question isn't whether AI will take our jobs. Maybe we're bleeding the people who do this work dry until they walk away, and then telling ourselves we have no choice but to hand the work to AI.

The same evidence pointed the other way

As the debate over AI acceleration continues, one argument keeps surfacing: but China. Slow down, the argument goes, and the United States falls behind its Chinese competitors. But this month's events show how much stranger that competition has become. The two sides aren't just racing each other. They're entangled.

On September 10, Anthropic published a threat report naming seven China-based labs it says used Claude to help build their own AI systems. Together, they ran about 190 million exchanges, or questions sent to Claude and answers sent back. Anthropic says the labs kept those answers and used them to train, or distill, their own models. Think of a student copying a stronger student's homework millions of times until they can imitate the work.

Moonshot, which makes Kimi, went further. So did DeepSeek. According to Anthropic, some user conversations were quietly forwarded to Claude, with Claude's answers returned under the labs' own names. Those exchanges were also kept for training. Users thought they were talking to Kimi, while some of their questions were being sent to an American company without their knowledge.

Beijing had already dismissed a similar set of accusations. A day before Anthropic's report, U.S. security agencies had issued their own advisory on distillation, and China's commerce ministry called it groundless, a common industry practice smeared as an attack. Distillation is a machine learning technique that trains a smaller, efficient student model to mimic the outputs, behaviors, or reasoning logic of a larger, resource-intensive teacher model.

Then, twelve days after Anthropic's report, China's internet regulator summoned all seven companies, focusing especially on DeepSeek and Moonshot. Alibaba ran by far the largest campaign, more than 151 million of those 190 million exchanges, and it wasn't the focus of the summons. That absence says something. Copying at scale was not the problem. The issue wasn't volume but what may have crossed the border. In one example, a Kimi user Anthropic assessed as likely connected to China's military, uploaded footage from hundreds of surveillance cameras in Chengdu and asked the system to flag unusual behavior by a person being tracked. Moonshot allegedly passed that material to Claude without telling the user.

Anthropic's complaint is about what left Claude. Beijing's is about what left China. None of this has been independently verified. But the same evidence, viewed from opposite ends, produces opposite grievances. Each side sees the part that matters to it.

Between two American companies, this would look like an argument about competition, scraping, or terms of service. Put a national border in the middle, and the same technical behavior becomes a question of national security, sovereignty, and who is allowed to learn from whom. What changes isn't the technology. It is who is doing it to whom.

Build faster, or build together

The argument finally arrived at the United Nations.

On September 23, the Security Council heard from Sam Altman, Dario Amodei, Yoshua Bengio, Hugging Face's Clément Delangue, and others about what happens when increasingly capable AI systems create risks that no company or country can see clearly on its own.

The meeting focused on what happens if an AI system does something unexpected. Who sees it? Who reports it? Who gets enough evidence to recognize that the same thing is happening somewhere else? The proposed answers involved more transparency, shared technical standards, incident reporting, and international coordination.

President Trump has offered a very different answer.

Nine days earlier, responding to calls for stronger AI safeguards, he wrote that the only guardrail AI needs is “a STRONG AND SMART (High IQ!) PRESIDENT.” His argument has consistently been tied to competition with China. Slow down too much, and China wins. The day before the Security Council meeting, he told the General Assembly the United States would reject any “globalist scheme” to control AI.

China's president has been saying almost the opposite. In July, Xi Jinping argued that AI's risks require answers from the entire international community. He called for greater risk awareness, keeping AI under human control, and a global governance framework built through the United Nations. He also warned against stretching “national security” too far in AI. It's worth holding that warning up against his own regulator's response to the Anthropic report detailed above.

There are plenty of reasons to question how either country's rhetoric lines up with its behavior. When more than 20 mostly European countries called for binding AI safety measures ahead of the General Assembly, the U.S. and China both abstained. Still, put the two statements side by side, and the disagreement is striking. One says the answer is national leadership: build faster, stay ahead, and trust the government already in charge to handle whatever goes wrong. The other publicly argues for coordination: share rules, manage risks together, and build institutions that can govern systems that cross borders.

Tying all these stories together highlights the question we all need to ask. When these systems behave in ways nobody expected, will anyone have enough evidence to understand what happened?

Because before we can decide what to do about AI, someone still has to be able to see what it is doing.

The Understory

After Hiroshima and Nagasaki, policymakers faced a problem deeper than simply getting countries to promise not to build atomic bombs. Even if countries agreed, how would anyone know whether they were keeping the agreement?

The authors of the 1946 Acheson-Lilienthal Report did not think occasional inspections were enough. An international organization standing outside the technology, periodically checking laboratories and facilities, would always be at a disadvantage. The people actually developing atomic energy would know more than the people trying to police them.

So they proposed the development of an international Atomic Development Authority that would control the most dangerous parts of atomic development, employ scientists, conduct its own research, license less dangerous activities, and maintain inspection powers.

When the idea became the Baruch Plan later that year, the reasoning was unusually explicit. The authority needed to remain at the forefront of atomic knowledge so that it could “comprehend, and therefore detect” misuse. Technical expertise wasn't something the regulator could draw on when trouble arose. In order to detect it, you needed to comprehend it.

The plan also did not view inspection and control as the same thing. When inspectors arrived from outside looking for violations, the system could be arranged to deceive them. The suggestion was to build an institution with sufficient technical understanding to know what normal looks like, control critical parts of the system, and recognize misuse when it occurs.

We talk about AI governance as a problem of rules. Transparency requirements, safety standards, incident reporting, evaluations, and international agreements. Eighty years ago, Acheson and Lilienthal were asking a harder question.

Does anyone outside the organizations building the technology understand it well enough, and see enough of it, to know when the rules are being broken?

See you next Wednesday. As always, my email is hello@wiobyrne.com.

Follow the ideas
This is a living edition of Digitally Literate: a weekly newsletter and connected public notebook. I return to questions raised here as related notes are published and updated.

Related Evergreens

• What Can People Do About AI? — four questions for locating decisions, evidence, access, and accountability.
• The Hugging Face Incident — what a cybersecurity evaluation that crossed into real infrastructure shows about AI systems, permissions, and oversight.

Previous: The Story About the Story · Full archive

More about my writing, teaching, and research at wiobyrne.com.