Greater than 10%.

ㅤ

Those are the odds of human extinction, in the next ten years, according to Anthropic's Evan Hubinger.

ㅤ

Given his credentials, I don't think anyone cares that he's speaking in his personal capacity and not on behalf of Anthropic.

ㅤ

He said this backing Jacob Coxon.

ㅤ

Coxon spent three years at OpenAI and was a core contributor to GPT-4o.

ㅤ

He joined Anthropic in July, quit yesterday, and left the industry altogether.

ㅤ

His take:

ㅤ

➤ Neither Anthropic nor OpenAI are acting responsibly,

➤ Both are racing toward self-improving superintelligence,

➤ Both are gambling with our lives.

ㅤ

Hubinger defended Anthropic in the same breath. They're trying their best, but they "do not yet have a plan to solve alignment for superintelligence."

ㅤ

He leads alignment science. At Anthropic.

ㅤ

Now read that again.

ㅤ

The person whose job is to prevent human extinction just said publicly there's no plan yet.

ㅤ

Meta, OpenAI and Anthropic all promised to halt development if the technology outran their control. Nobody has paused.

ㅤ

Senator Bernie Sanders and Congressman Greg Casar are writing a law to apply a brake. Creation of a new cabinet-level agency, twenty years in prison for individuals, and forced dissolution for firms.

ㅤ

Casar's line: cutting-edge AI is less regulated than the average food truck.

ㅤ

Here's what the researchers like Coxon actually asked for:

ㅤ

➤ Coordination between labs

➤ A temporary pause on capability gains

➤ Independent review. Someone not employed by the labs checking the work.

ㅤ

1,400 researchers signed that letter in July.

ㅤ

Unglamorous stuff.

ㅤ

Also the only version where the people who understand AI are the ones setting the limits.

ㅤ

I've been building AI products since 2014. I believe AI is one of the greatest tools for human flourishing ever built.

ㅤ

And I also think the boring version is still available.

ㅤ

It stays available only if the industry picks it first.