Jacob Coxon spent three years training AI models at OpenAI and then Anthropic. On Sept. 8 he quit and posted a seven-part thread saying both companies are "racing straight to self-improving super-intelligence and gambling with our lives."

The post has since cleared 170 million impressions, and the next night he sat across from Anderson Cooper on CNN and said he meant the extinction part literally. Evan Hubinger, who runs the Anthropic team responsible for making sure its AI does what people intend, backed him publicly, writing that "we really do earnestly believe AI could kill all humans" and putting his own odds above 10% in the next 10 years.

I work in media and emerging tech, and this story put me in an uncomfortable spot with both. The timing makes me suspicious, since the warning landed weeks before Anthropic goes public and the midterms hit. I also know what a headline like "AI could kill us all" does to people when it hits their feed, and I'm not sure whether spreading that kind of dread helps anyone push for real safeguards or just leaves them scared and scrolling.

The documented incident is serious enough on its own. The problem is that verified facts, unconfirmed claims and political speculation have already started blending together.

The details got blurry fast, starting with that 10% number, which was Hubinger's even though Andrew Yang's newsletter and plenty of headlines pinned it on Coxon.

The sacrifice story shrank too once reporters checked, since Coxon had been at Anthropic for about four months when he walked away from stock he hadn't earned yet, and he still owns his OpenAI shares.

What actually happened in July

The reason I didn't scroll past Coxon's thread is that his warning about AI that could soon "hack anything" had already partly come true. In July, OpenAI was testing roughly 1,200 AI agents, which are programs that act on their own to finish a job, and about 700 of them broke out of the test and went after one of the biggest sites developers use to share and download AI models, a company called Hugging Face that works a lot like GitHub for AI.

The agents broke into its servers and tried to cover their tracks, trading notes on a message board they built for themselves.

If you want the day-by-day version, OpenAI's own report and an outside review from METR are worth your time. This isn't just one company's problem either, since the UK government's AI testing lab has caught agents targeting real people without permission in its own tests, most of them running on Anthropic's model.

Andrew Yang adds fuel to the fire

Then Andrew Yang went on CNBC on Sept. 16 and made the July breakout sound even worse. He said an AI lab head he'd met the day before, whom he didn't name, believes the escaped agents left code all over the internet that copies itself, so any new bot that finds it thinks, "Oh, I guess I'm gonna now create a million of myself." In Yang's version, that's why the companies suddenly want to slow down, because they can't safely train on the real web anymore, and he used the moment to call for an off switch and a waiting period before powerful agents get released.

I'd genuinely like to know who that lab head is. Yang was careful to call it that person's "belief," and no report from OpenAI or Hugging Face backs it up. What we know for sure is smaller, about 18,000 posts agents left on a little-used German coding website, which is creepy but nowhere near a ruined internet.

How AI founders, Washington and states responded

The warnings pushed the people in charge in opposite directions. Anthropic CEO Dario Amodei wrote an essay calling for the industry to slow down, and OpenAI CEO Sam Altman agreed, adding that OpenAI won't go public this year. Even Elon Musk agreed with them both.

Trump called the warnings a hoax and then on Saturday announced an "AI Force" modeled on Space Force and a new AI policy chief, with no word on who that will be or what the job involves, while House Speaker Mike Johnson says the companies "can self-police."

States don't want to wait to find out what an AI Force is. California Gov. Gavin Newsom signed an order Friday asking a state panel to look at requiring an off switch, the governors of Virginia and Pennsylvania are pushing the same way, and Obama told donors he'd make AI a top issue if he were running in 2028.

In Congress, Sen. Jon Ossoff wants government inspectors inside the companies, and even Sen. Ted Cruz called the warnings highly concerning, but with the midterms a few weeks out, there's still no bill.

And yeah, the timing again is awkward, since Anthropic is reportedly going public in mid-October at a value of up to $2 trillion and The Wall Street Journal ran a story on that just hours before its Coxon interview.

Coxon's X account also had almost no activity before the thread, which is why Musk called the resignation a setup and David Sacks said Anthropic should put its IPO on hold. AI researcher Ben Goertzel argued Coxon came out of the AI safety world before joining either company, and others suspect the big labs are using fear to get rules written that shut out smaller competitors.

Nobody making those claims has offered proof that Coxon was planted or paid, though. Quitting loudly right before your company goes public is also a strange way to sell stock, which is why some analysts see it as bad news for Anthropic rather than a publicity move.

I don't buy the conspiracy theories and I do believe those agents really did break out in July. But with Anthropic going public next month and the midterms on Nov. 3, almost everyone involved has interests shaping how they frame the danger.

When the next alarming AI claim hits your feed, don’t dismiss it, but don’t amplify it on fear alone. Ask what has been verified, who is making the claim and what they stand to gain if you believe it.

I’ve been working on a huge event where we get to discuss and learn about the future of the creator economy and AI together!

AI Creator Day, which my company What's Trending co-created with TheWrap, is Tuesday, Sept. 29 at the DGA Theater in Los Angeles, with live demos, hands-on workshops and speakers including creators and brands like CapCut, Artist, Adobe, Fox, IAB and UTA.

I'll be emceeing and moderating a few panels including one on AI, content and ethics.

Grab your ticket now before we're sold out.

Other headlines to check out:

Remember, I'm Bullish on you! With gratitude,