OpenAI’s AI broke out. It’s time for digital disaster planning

You’ve probably heard about the OpenAI hacking incident—where an AI agent escaped its testing confines and hacked a website on the open web.

When PCWorld staffers discussed this development, I was actually surprised by the reactions. One person described it as “the most terrifying security and AI news I’ve heard in recent memory.” In reverse, they seemed surprised by my relative calm.

Don’t get me wrong. I’m not unaffected. But I don’t think of AI as the problem.

AI is a tool. Whoever wields the tool sets the agenda. In this case, OpenAI ran a benchmark specifically to evaluate how well its new models can find and exploit vulnerabilities. And in its own way, the AI agent being tested did exactly that. It found an unknown vulnerability in its controlled environment, broke out to the open web, and hacked into a website called Hugging Face (a code repository for AI developers).

I don’t find it scary OpenAI’s AI agent basically chose to cheat on its test. AI isn’t human. It also doesn’t operate independently, even if marketed as such. As Olivia Buzek, Staff AI Engineer at IBM said in a podcast: “Fundamentally…models by themselves cannot escape containment. They can only do the things that you give them the tools to do. So what that means is, you need to be very careful about what sort of tools you hand it.” In this context, her reference to tools is about the type and level of access developers give to AI models.

Humans make AI. Humans determine how AI is configured. I’m much more concerned about the people developing AI. They are learning in real time the consequences of automating tasks and processes at dramatically bigger scale and speed. But they seem unprepared to protect the rest of us as mistakes happen.

Instead, AI companies have stayed quiet about the uglier parts of development. Hugging Face brought this breach to light, not OpenAI. OpenAI identified itself as the source five days later. Meanwhile, rival Anthropic just revealed it too has seen Claude hack live websites—sharing after the fact and as OpenAI dominates headlines.

So what scares me is splash damage. I can see a future of consumers dealing regularly with the consequences of human decisions around AI design. We already can’t control the number of attacks on businesses, which lead to data leaks and other online security issues. Matters will worsen dramatically in a world where AI agents run amok, either accidentally or purposefully. AI models can continually hammer at a task without fatiguing.

We of course as everyday users still have our voices, and we should use them. We should ask our government representatives to look deeper into regulation, to establish a structure for AI incident reporting and remediation. Only part of the tech industry’s major players have committed to open source solutions; OpenAI, Anthropic, and Google are conspicuously absent. Problems cannot be solved if kept secret.

We also need to be prepared for disaster. What would be considered unexpected and catastrophic now could become usual. This OpenAI incident has a similar vibe as the early days of the internet. Just think back to when a graduate student released a computer worm on the open web—without any intent to cause the subsequent mayhem and financial damage. 

As AI development bumps along, I can picture situations where AI amplifies human error to an extreme. Denials of service at massive scale. Wholesale, abrupt account lockouts in the millions. Etc.

Online security ultimately boils down to access, in my book. Those who should have it, do. And those who shouldn’t, don’t. But when that fails, you need a backup plan. You need a way to get yourself out of a sticky spot when established systems fail you. (Or at least, a way to cushion the impact.) So: Are you ready if your bank account were to become inaccessible? Your electricity shut off because your account was erroneously flagged as delinquent? Losing access to an important email, messaging, or social media account because its security was breached or the service was forced to go offline?

We tend to think of disaster preparedness for natural events—hurricane, flood, fire. But I think this OpenAI hacking incident is a warning about digital disasters. A harsh, cold slap of a new reality, where disruption to online systems will happen in just minutes. Whether such instances leave behind major destruction is up to AI developers. I’m not encouraged by their approach so far.

(Just as I finished writing this, OpenAI says maybe yet other AI agents escaped their sandboxes, too.)

In the news

OpenAI of course turned up in headlines repeatedly this week—not only did Hugging Face release a detailed report on how the attack on their site unfolded, but it turns out the same AI agent hacked a few other sites in addition to Hugging Face. And as mentioned just above, other AI agents may have snuck out on to open web, too.

But unfortunately, that wasn’t the only concerning news. Microsoft, Adobe, and even the Vatican fumbled on privacy and permissions, with the info only coming to light now.

The bad

• Older versions of the Adobe Acrobat extension for Chrome could leak your private WhatsApp chats, due to a flaw. This issue has since been patched, so make sure you’re running version 26.5.2.3 or newer.

• For more than six months, the Vatican’s official “Click to Pray” app leaked data of over 700,000 users, including names and email addresses. Users should screen email and messages carefully, especially any asking for money.

The concerning

• Microsoft apparently tags Windows devices with a unique identifier—and then can link activity performed on that device back to that ID. Jokes about hackers who don’t use Linux aside, the privacy implications still matter for us everyday Windows users.

• Your car could be vulnerable to hackers. Newer vehicles may have an exploitable aftermarket car security system installed, even if you never asked for one. Check to see if yours does; if yes, either update it to the latest firmware or remove it all together.

• Claude can now hook into 1Password to log into services on your behalf—a feature that my coworker Ben Patterson took for a spin just before the OpenAI hacking debacle. This level of trust in AI feels particularly risky, so if you must use it, consider linking to a second account that holds just the passwords Claude would need.

Tip of the week

Like your privacy and live in California? You can request deletion of your information from data broker databases, which create profiles on individuals that include full legal names, known addresses and phone numbers, social security numbers, relatives, and more. Data brokers otherwise openly sell access to these details.

To get started, head to the official DROP website, which is free and managed by the state of California. Just one request applies to all data brokers registered with California (and they must register in order to avoid fines).

August 1 marks the start for when data brokers must begin deleting profiles. So if you’ve been waiting to put in your request—or to let your state know that you want a similar program, being a non-Californian—now’s the time.