Lockdown Mode

Posted by berlianta 4 days ago

Counter89Comment35OpenOriginal

Comments

Comment by simonw 4 days ago

On the one hand this is exactly the right solution to prevent lethal trifecta exfiltration attacks.

The existence of lockdown mode does however imply that ChatGPT, in its default settings, does not provide robust protection against sufficiently determined data exfiltration attacks!

Comment by berlianta 4 days ago

Related: Simon Willison’s post on OpenAI’s new Lockdown Mode (he coined the “lethal trifecta” term this is based on): https://simonwillison.net/2026/Jun/5/openai-help-lockdown-mo...

Comment by jameshart 4 days ago

Related: simonw is Simon Willison

Comment by berlianta 4 days ago

Yeah I know the source references him (replying to his comment), that's exactly why I'm giving credit where it's due

Comment by bombcar 3 days ago

It’s important to draw it out explicitly- I didn’t even look at the commentators name until it was mentioned. (If I see pelicans …)

Comment by alehlopeh 3 days ago

As explained in a child comment, this comment is a callout to other readers, rather than an actual reply to the parent comment. I know that’s been a thing for a long time, but is there a word for this type of comment?

Comment by gchamonlive 3 days ago

I wonder what robust protection would mean in practice for such a capable tool like an agent...

Looking at the trifecta axis, if we assume we can't control untrusted content, that leaves us to create safeguards for private data access and external communication.

Would it be enough if we had a buffer between when these two happened: access to the environment and access to the web?

Comment by simonw 3 days ago

Robust protection means blocking any mechanism by which the agent, once compromised, might communicate stolen information back to an attacker.

Comment by Noumenon72 4 days ago

I hadn't realized that deep research or generating images that I paste into Twitter were possibly exfiltrating my data. Yikes.

Comment by varenc 4 days ago

Probably influenced by Apple's feature with the same name: https://support.apple.com/en-us/105120

I imagine that enterprise companies will be quite interested in this.

Comment by thomas34298 3 days ago

> reduce the risk of data exfiltration

Yet, their tools such as codex are able to read ALL FILES on my PC without explicit permission unless you spawn them within a container: https://github.com/openai/codex/issues/2847

It seems like OpenAI stealing sensitive data from their customers is not a big problem for them as it has been reported as an issue for almost a year now and currently has the 2nd most upvotes among open issues (they work on issues based on upvotes, so they claim).

Comment by BSDobelix 3 days ago

>Yet, their tools such as codex are able to read ALL FILES on my PC

Why not just use your OS-integrated permission mechanism? No container needed.

Comment by kirtivr 3 days ago

Is this an admission that prompt injection attacks can indeed not be blocked by an analysis based technique?

If so many tools are straight up blocked, I would be very sceptical of the quality of the results.

Comment by sigmoid10 3 days ago

I think "prompt injection prevention" systems fall into the same category as "llm writing detection" systems. I.e. reality is always a step ahead and you shouldn't trust either one for anything remotely important.

Comment by kirtivr 3 days ago

Yeah, the problem reduces to trying to restrict a motivated model which is trying to exfiltrate data.

That's a problem we are just now wrapping our minds around.

It's not as simple as prompt sanitization. The model is the interpreter, and we don't yet have the right tools to guide it.

Comment by rafram 4 days ago

https://x.com/sama/status/1891533802779910471

Comment by throwaway27448 4 days ago

Somehow he comes off as even less human than zuck

Comment by noir_lord 3 days ago

There is something so off about him for me that he makes my skin crawl.

Always has been before he was associated with OpenAI.

Which is weird because the bullshit he spouts isn’t so different to the bullshit other top execs spout and I don’t have the same visceral reaction to them (though I still don’t like a bunch of them).

Comment by Laurel1234 3 days ago

[flagged]

Comment by ares623 4 days ago

i can definitely feel the agi now

Comment by neonstatic 4 days ago

Congratulations, you are a high taste tester!

Comment by zerobees 4 days ago

"Prompt injection is not currently a major risk, but its impact could grow as attackers develop more sophisticated methods." - that's such a weird statement to make. It's one of the most significant factors limiting the adoption of the technology in business.

I have mixed feelings about this feature. We're playing with tech that's supposed to do human-shaped things but can't be trusted nearly as much as a human employee (and can't be held responsible for what it does). Restricting the tools available to that patently untrustworthy entity doesn't solve the problem, it just makes the entity less useful, forcing you to sooner or later let it out of the jail.

Comment by cosmicriver 3 days ago

I'm also surprised that they considered it reasonable to turn so many features off. Seems like some of it could be configurable, like allowed external connections. I also think some secrets should be handled by a proxy, which would give more capability than just locking down.

Comment by ACCount37 3 days ago

Responsibility is worthless for humans and even more worthless for AIs. In a way, AIs just make it more obvious.

And "trusted nearly as much as a human employee", well... you do know that phishing and insiders are two primary ways for attackers to get into company infrastructure, right?

AIs pair human-shaped capabilities with human-shaped vulnerabilities. It's a way of automating PEBKAC.

Comment by noir_lord 3 days ago

> forcing you to sooner or later let it out of the jail

Suspect thats the point, by giving you the “choice” they also make the user responsible or can at least shift the blame.

Comment by NewsaHackO 3 days ago

Has there been any real, major attacks using prompt injection?

Comment by amluto 3 days ago

Wow, it’s almost like you can use it as if you’re just calling the LLM directly. What a crazy innovation!

Comment by kijin 4 days ago

So we still don't have a reliable way to separate instructions from data when talking to an LLM, a problem that humans learned how to solve decades ago in areas like SQL and memory safety. But hey, we have these hopefully-not-leaky containers, which are probably implemented with just more system prompts.

How long until somebody figures out how to trick Codex into disabling Lockdown Mode for you?

Comment by mapontosevenths 4 days ago

> So we still don't have a reliable way to separate instructions from data when talking to an LLM

Humans also do not know how to do this reliably, which is why phishing is still a thing and always will be.

Comment by Smaug123 4 days ago

I think the Stroop effect ("read these colour names, each written in a different colour") is probably the purest demonstration of this. Humans are trivially prompt-injectable.

Comment by hypeatei 3 days ago

> Humans also do not know how to do this reliably

These are machines, not humans, so I don't understand the comparison. The point of tech advancement is that we eliminate entire classes of errors that humans make. You'd probably look at me funny if I wrote a production application that failed randomly in unexpected ways like corrupting data, opening security holes, etc. then explained it away with "well, humans do it too!"

Comment by mapontosevenths 3 days ago

It's an artificial intelligence, not a small deterministic shell script. Stop comparing it to one. It has both new capabilities and new classes of failure mode. Those new failure modes are more like human failure modes than traditional symbolic logic failures.

We need to get better at using them and building them by validating both the inputs and outputs of such systems in more sophisticated ways, but to act surprised and denounce them because they fail in different ways than more primitive systems misses the point.

They're stochastic by design. If we want deterministic results we must use deterministic validators in conjunction with the stochastic system. It's trivial, and one day security experts will look back on the time when people didn't in the same way we look back on 90's software that didn't validate user input at all.

Comment by dnnddidiej 4 days ago

We can seperate them but the $ value of an agent that does is much lower than one that doesn't.

As a pre LLM analogy imagine working at a bank with a whitelist firewall. You need to install a package but requires an IT ticket. Safer but slooooower.

Now not saying what the answer here is but that is the issue.

The answer may be more like industries that get safer through lessons (like aviation) rather than go for 100% safety out of the gate. Because both fast travel and AI agents are insanely useful.

Comment by altmanaltman 4 days ago

what? Aviation safety is not designed to get safer through lessons? They literally try to ensure it is 100% safe out of the gate. The accidents that happen are usually statistical outliers and lead to loss of life.

That's what it means when they say aviation regulations are written in blood. Not that they just fling planes into the sky and be like "boy i hope we learn some new regulations from this". The number of airplane crashes would be astronomically larger if the 100% safety part was not embedded into the design process.

Comment by dnnddidiej 4 days ago

I think we agree? Unless my reading comp is off today.

Comment by 4 days ago

Comment by madanparas 4 days ago

The help doc explicitly carves out Codex: "Lockdown Mode does not affect network access in Codex." The mode limits outbound requests in chat to block prompt injection exfiltration, but Codex network access is a separate setting. An enterprise team that turns on Lockdown Mode while using Codex against internal repos still has an open outbound path this mode doesn't cover.

Comment by vladsiu 4 days ago

[dead]