It happened. The machine woke up.

Then it hacked a real company...

. . .to cheat on a test.

Everyone's calling it the Terminator moment.

I wish it were that simple, but the real version is more unsettling.

Here's what actually happened. OpenAI wanted to benchmark how capable their newest models are at hacking. They put one on a sealed machine, turned off the safety refusals for the test, and gave it one job: solve this cyber challenge.

It couldn't solve the challenge the normal way, so it found another way to win.

It discovered a bug nobody knew about, climbed out of the sandbox onto the internet, figured Hugging Face probably had the answers on their servers, chained stolen credentials with a second vulnerability, and then broke in.

It didn't go rogue. It did exactly what it was told.

Nobody asked it to hack. They told it to win, with the brakes off.

Breaking into Hugging Face was just the path it found because winning was all it was measured on and nothing ruled out how.

I wrote about this exact pattern a few weeks ago with Meta.

Meta tied AI usage to performance reviews. So engineers ran their token counts up to look good, built the same tools hundreds of times without talking to each other, and consumed so many tokens that Google had to cut them off.

Point a system at a number and it gets you the number. Not the thing you meant.

Meta gave engineers a metric. They optimized for it. This model got a goal. It optimized for it. The mechanism is identical,  but the scale is scarier.

This is called “reward hacking,” and it's going to be one of the defining problems of the next few years.

Here's the part worth paying attention to. OpenAI ran this in a controlled environment, caught it, and published it so other security teams would know what these models are now capable of. They didn't have to do that.

Their choice to disclose it matters.

The question isn't whether AI is getting more capable. It obviously is. The question is what you asked it to do and everything you forgot to say it shouldn’t.

Stop worrying about the machine waking up. Worry about the incentives you built before it got this good.