Trusted access for the next era of cyber defense

Posted by surprisetalk 6 days ago

Comments

Comment by alopha 6 days ago

That's a lot of waffle to try and say 'we've got a really scary next model coming too real soon, promise!'

Comment by guzfip 6 days ago

More like they realized how much money they were wasting letting the proles generate slop and vibe code the same CRUD app they rewrote in 5 different JavaScript frameworks a few years back.

The money is in enterprise and government. The consumer market doesn’t remotely pay enough. It’s just the same story with Microsoft purposely making Windows an unusable mess because that’s not where they make their money. It was good to establish themselves, but that market is getting dumped.

Comment by flyinglizard 6 days ago

Wait six months, get the Chinese version.

Comment by NitpickLawyer 6 days ago

The move towards "trusted partners" also acts as a way to protect from distillation.

Comment by everlier 6 days ago

Changes as we speak, z.ai is the first one to show differential pricing

Comment by Avicebron 6 days ago

I don't think they've added enough cyber. My cyber workflow demands more trusted access for cyber so that I can use these cyber-permissive models for my cybersecurity.

Comment by Jedd 6 days ago

It's a source of minor, but persistent, annoyance that security people have tried to abscond with the prefix cyber, morphing it into a synonym for security.

Having grown up reading cyberpunk novels about life in cyberspace, a passing interest in cybernetics (though not of the Sirius Cybernetics Corporation variety), it's frustrating to lose a 'this means computer or internet related' prefix.

Comment by bee_rider 6 days ago

Hmm, I guess this puts the unregulated banking enthusiasts’ stealing of the crypto prefix in a new light.

Comment by SturgeonsLaw 5 days ago

As far as I can tell, using the word cyber to specifically and only talk about security has come from the kind of suits who take Gartner seriously.

I don't know any techies who use the term like that, unless they're in a role that interfaces with the suits.

Comment by ofjcihen 6 days ago

Whoa hey now, if they just give out all the cyber all at once they might run out or worse, the bad guys will horde all the cyber for themselves!

No no, best to have them distribute the cyber to us responsibly.

Comment by SoftTalker 6 days ago

Just wait until you meet the Cybermen.

Comment by TeMPOraL 6 days ago

I just hope they're the Responsible Cybermen.

Comment by swyx 6 days ago

you make fun of it but i kind of like that the security community has just embraced this kinda old school hokey term. its a short hand. leave them be.

Comment by cshimmin 6 days ago

Incidentally, I recently learned the origin of the term. Cyber - short for cybernetic - is from the greek κυβερνήτης (kybernetes), meaning helmsman. The original use of cybernetics is in the context of automated control systems, so steering a rudder was a good analogy. It is also the origin for the name k8s.

Comment by twoodfin 6 days ago

In the early days of socialization on the Internet it had a very different meaning!!

Comment by TeMPOraL 5 days ago

In my headcanon, I still read k8s as "network of cubes", as in Borg cubes, as Kubernetes itself is a poor man's Borg (as in the thing that Google runs on, named after Star Trek Borg, known for cube-shaped ships referred to as "Borg cubes"). The whole kyber thing sounds like an explanation after the fact, to detach from the Collective legacy.

Comment by Melatonic 5 days ago

Have they ever released the full internal Borg toolset and software ?

Comment by chickensong 6 days ago

a/s/l?

Comment by 5 days ago

Comment by 6 days ago

Comment by atoav 6 days ago

Just make sure you use cyber periphery (e.g. a cyber keyboard) to type out your cyber prompts and you will be cyb.. ahh.. fine.

Comment by zarzavat 6 days ago

I'm ready with my robe and wizard hat, Sam.

Comment by FacelessJim 6 days ago

I would definitely love a glass of smoked cyber

Comment by tb0ne1521 5 days ago

[dead]

Comment by ofjcihen 6 days ago

I love that in the era of having LLMs summarize everything all of these companies have opted for what I call the “YouTube streamer apology video” tone and length for these announcements.

These feels more or less like a way to get in the news after Anthropic's Mythos announcement by removing some guardrails. I’m still signing up though.

Comment by 6 days ago

Comment by bunnywantspluto 6 days ago

It seems like local LLMs will get popular for cybersecurity if this trend of locking access to models continues.

Comment by alephnerd 6 days ago

Not really. Not performant enough. Most organizations who would be interested in using a foundation model for security would either purchase the model directly or purchase a vendor who adds their special sauce or context to the model

Comment by gavinray 6 days ago

I completed the "Trusted Access" verification, but it seems to have unlocked nothing in the OpenAI API or Codex models.

Just FYI for others.

Comment by hoss1474489 6 days ago

I see a Security button in the what’s new box in the Codex section of the ChatGPT website. It appears to allow me to run vulnerability scans against my connected GitHub repositories.

Direct link: https://chatgpt.com/codex/cloud/security

Comment by gavinray 6 days ago

I also have access to this but can't be certain if it was there before or not.

Anyone else who hasn't verified able to access?

Comment by Kye 5 days ago

404 for me

Comment by alphabettsy 6 days ago

That’s been there for awhile.

Comment by ofjcihen 6 days ago

So it seems like you just…have it once you get approved. I’m testing it now and nothing indicates I’m running a different model but it just doesn’t fight me on cybersecurity stuff

Comment by mikewarot 6 days ago

It's important to keep perspective, the holes that everyone (including LLMs now) keep finding in pretty much everything are mostly the fault of running things with ambient authority, instead of using systems based on default deny, and capabilities.

I used to think we were 20 years away from a shift to Capabilities based Operating Systems, which were ----> this <---- close to being adopted widely when the PC revolution swiped them aside.

Unfortunately, I think we're about to repeat history, and we're now 20+ years out from actually solving things, AGAIN. 8(

Comment by NoahZuniga 5 days ago

Many (maybe even most bugs) the ais are finding are memory safety errors, which is pretty clearly not "the fault of running things with ambient authority". The data is treated as untrusted, but due to a mistake can still do something it shouldn't.

Comment by mikewarot 4 days ago

The thing is, if you're only allowed to access the actual files the user selected (at runtime) those are the only things that could possibly be corrupted. A memory error in any given app doesn't set up for a system compromise in a capabilities based system.

Comment by TeMPOraL 5 days ago

"Solve things" or actually do something useful, pick one.

If anything, maybe the security community can finally be arsed to consider ad-hoc delegation of authority as a core concept and a basic use case, because that's arguably the primary source of persistent user-level security issues in computing.

In real life, it's absolutely normal to ask random people on the fly to do something in your name, with your credentials - whether that's sending your kid with your credit card for a grocery run, asking spouse to do some bank transfers for you or set up a new computer for you, or asking a co-worker to operate some system. It's the other reason people write passwords on post-its: even without bullshit password strength rules (see xkcd://936), there's still a frequent need to share passwords with people.

Meanwhile, for the past decades, security community has been insisting on tying authority to individuals, and doing everything possible both technologically and socially to prevent authority delegation (except in top tier corporate systems, where this is technically supported, but in such convoluted, complex and broken ways that it may as well not exist - people will still resort to post-its in drawers).

Until this basic concept is recognized, I fear more broad security improvements will only result in more useful work being prevented from happening, and more people-years wasted as users figure out ways to defeat security measures so they can do their actual jobs.

Comment by mikewarot 5 days ago

In the 1970s my Mom would give me a $20 capability token to exchange in a store for cigarettes, in her name, while she waited withy sisters in the car. I was about 15 at the time, and it was a normal part of life. She never gave me a signed blank check.

Giving $20 to an AI is far safer than giving it your credit card. The effects are limited to $20 of loss.

Comment by TeMPOraL 4 days ago

Right. But there's almost no software that supports the equivalent of limited capability tokens, much less casually handing them around. In contrast, in real life, it's a common use case, and we don't usually even bother with capability limits, because it's too much hassle - we rely on trust (part of which is persistent relationship that continues beyond current interaction) + spatial proximity and temporal limits.

I.e. even if your mom handed you her credit card, she was still there in a car nearby (spatial proximity), and was waiting for you there (temporal limit), and she was your mom (persistent trust-based relationship), which is sufficient protection from the risk of you running away and spending her money on hookers.

(How you managed to buy cigarettes as a 15yo is beyond me - or maybe there were no age checks in 1970s yet?)

Coming back: in real life, we don't bother with restricting the access tool, everyone is transiently giving much more access than they need to random things, and expect them to not abuse it. Meanwhile, cybersecurity is mostly stuck in the mindset of passwords being your identity, and being like underwear (change frequently, don't share), and the concept of delegation of authority doesn't exist beyond some enterprise systems. Which is why, in real world, everyone says "fuck it" and just shares passwords as needed.

Comment by Melatonic 5 days ago

Are those really things people do all the time ? Not sure I would trust any kid with my credit card

Comment by TeMPOraL 5 days ago

I didn't say any kid, but rather one's own kid - whether they can or cannot be trusted is mostly a parenting issue, so different people have different experiences.

But yes, me and my siblings would often do grocery runs for our mom, with her card in hands, and I also think nothing of handing my own card to my wife (who already knows the PIN), or mine or her siblings, or even some acquaintances, because I trust them.

The larger point (even larger than my previous comment) is that, contrary to what cybersecurity (and especially cryptocurrency aficionados) community believes, the real world runs on trust. Trust is not a bug, it's a feature - an optimization that makes societies and civilizations scale. Trust has its own limits and structural complexities, it has its ebbs and flows, but it's absolutely vital and systems that do not support it (or try to eliminate it) simply gets worked around. Not out of spite, but out of necessity - otherwise nothing would ever get done.

Comment by 5 days ago

Comment by iammjm 6 days ago

"trusted" + openai just simply doesn't compute for me any more

Comment by greatgib 6 days ago

All of that reminds me about how gpt2 was almost too dangerous to be released to the world...

Comment by onoesworkacct 6 days ago

It was. The internet really has been filled with abject shit and social media is bots talking to bots.

Comment by keyle 6 days ago

It's not like the world is a better place since...

Comment by mmooss 6 days ago

This approach means only a tiny portion of the population will every qualify. Doesn't that make everyone else beholden to those few, who are beholden to OpenAI?

Another solution is to make software makers responsible and liable for the output of their products. It's long been a problem that there is little legal responsibility, but we shouldn't just accept it. If Ford makes exploding cars, they are liable. If OpenAI makes software that endangers people, it should be the same.

> Democratized access: Our goal is to make these tools as widely available as possible while preventing misuse. We design mechanisms which avoid arbitrarily deciding who gets access for legitimate use and who doesn’t. That means using clear, objective criteria and methods – such as strong KYC and identity verification – to guide who can access more advanced capabilities and automating these processes over time.

KYC isn't democratic and doesn't prevent arbitrary favoritism, it's the opposite: It's used to control people and to favor friends and exclude enemies.

Comment by sureMan6 6 days ago

> Another solution is to make software makers responsible and liable for the output of their products. It's long been a problem that there is little legal responsibility, but we shouldn't just accept it. If Ford makes exploding cars, they are liable. If OpenAI makes software that endangers people, it should be the same.

That kind of thinking is exactly why LLMs are so censored, because people think OAI should be liable if someone uses chatgpt to commit cyber crimes

How about cyber crimes are already illegal and we just punish whoever uses the new tools to commit crimes instead of holding the tool maker liable

This gets complex if LLMs enable children to commit complex crimes but that's different from just outright restricting the tool for everyone because someone might misuse it

Comment by 0x3f 6 days ago

There's always some wedge issue that means "don't punish the toolkmaker" is not politically viable. You can pick from guns to legal drugs to illegal drugs to all kinds of emotive things.

And once the wedge is in and the concept of maker responsibility is planted, it expands to people's pet issues, obviously.

The actual line of who gets punished just ends up at some equilibrium in the middle. Largely arbitrarily.

Comment by kaashif 6 days ago

I think the classic one is pedophiles and protecting children.

If someone uses ChatGPT to create child porn or worse, to get help tracking down and meeting children, there is NO way in hell the public will accept "don't punish the toolmaker" as a principle.

Comment by marshray 6 days ago

"It's just a neutral tool" gets a lot harder to claim once a vendor starts specifically training and marketing the model for its ability to bypass security controls.

Yes, pentesting tools, even automated ones, are often legal. But they commonly do run up against legal restrictions and risks. They're marketed very differently from ChatGPT.

Comment by luma 6 days ago

So who is at fault in your solution, the org who created and shipped the software bug, or the company that discovered it?

I don't see how OpenAI is Ford in your analogy as OpenAI didn't make the software that blew up.

Comment by Havoc 6 days ago

>democratized access

>partner with a limited set of organizations for more cyber-permissive models.

I get where they're going with this, but still rather hilarious how they had to get a corporate speak expert pull of the mental gymnastics needed for the announcement

Comment by 0x3f 6 days ago

It must be representative democracy! And our representative is... Larry Ellison. Oh no.

Comment by TeMPOraL 5 days ago

No need for corporate speak experts anymore, SOTA LLMs are more than capable of doing that job now.

Comment by nullc 6 days ago

Make cyber not cyber.

Comment by 2001zhaozhao 6 days ago

Requiring verified access is a good idea to mitigate risks from hacking while still giving people access to the latest models. Take notes, Anthropic.

Comment by striking 6 days ago

A 5.4 spin with slightly different guardrails is not "access to the latest models". We know this to be true from the article because they have a section entitled "Looking ahead to our upcoming model release and beyond". I wonder if they didn't just feel like they were caught out by Mythos.

Comment by zb3 6 days ago

> Ultimately, we aim to make advanced defensive capabilities available to legitimate actors large and small, including those responsible for protecting critical infrastructure, public services, and the digital systems people depend on every day.

Translation: we aim to make defensive capabilities available to US and their vassals so they can protect critical infrastructure, while ensuring countries that are independent can't protect against US attacking their critical infrastructure.

Fortunately, this plan will backfire - the model capability is exaggerated and these "safeguards" don't reliably work.

Comment by CompoundEyes 6 days ago

Wonder if Cyber would’ve caught the Claude Code source map leak?

Comment by rishabhaiover 6 days ago

I mean Anthropic clearly wins with the name (Mythos vs 'GPT-5.4-Cyber')

Comment by Phelinofist 6 days ago

Sounds totally reasonable to trust OpenAI and the sociopath sama.

Comment by realisticid 5 days ago

[dead]

Comment by spacebacon 6 days ago

[flagged]

Comment by ACCount37 6 days ago

First, it looks like an "AI psychosis" paper. AI psychosis has been going through armchair philosophers the way crack was going through the low income neighborhoods back in the 80s.

Second, it does not look relevant to the discussions in any way, fashion or form.

Comment by spacebacon 6 days ago

If I didn’t empirically prove it I would agree with you. Computation and semiotics are two fields that want nothing to do with each other. The SRT is through several stages of quantitative validation. For the first time in 150 years semiotics is not a philosophy. It is proven to have computational value. The SRT bolts onto any model and improves it. I’ll be sure to link you to the benchmarks when published. The relevance is it makes these treasured black boxes irrelevant.

Comment by ACCount37 5 days ago

Bold fucking claims for a "paper" that: makes an LLM with an awkward architectural tumor, and proves that it doesn't completely die on a purely synthetic task.

Further than most "AI psychosis" papers go, but still not in any way far.

And "makes these treasured black boxes irrelevant"?

With wild claims like this, either demo a generational improvement on a live model or GTFO.

Comment by spacebacon 5 days ago

I’ve been here over a decade longer than you sport. No need to bully people out when you are only 8 months in. I will be updating here when the model is live. Expect no further engagement.

Comment by spacebacon 5 days ago

[dead]

Comment by ACCount37 6 days ago

Too little too late. OpenAI's shit was nearly worthless for cybersec for what, a year already?

ChatGPT 5.x just tries to deny everything remotely cybersecurity-related - to the point that it would at times rather deny vulnerabilities exist than go poke at them. Unless you get real creative with prompting and basically jailbreak it. And it was this bad BEFORE they started messing around with 5.4 access specifically.

And that was ChatGPT 5.4. A model that, by all metrics and all vibes, doesn't even have a decisive advantage over Opus 4.6 - which just does whatever the fuck you want out of the box.

What's I'm afraid the most of is that Anthropic is going to snort whatever it is that OpenAI is high on, and lock down Mythos the way OpenAI is locking down everything.

Comment by jruz 6 days ago

That’s the whole point of this variant of the model, it won’t have those guardrails.

Comment by ACCount37 6 days ago

Yes. But "perform a humiliation ritual of KYC to access the actual model instead of the nerfed version of it that's so neurotic about cybersec you have to sink 400 tokens into getting it to a usable baseline" does not inspire any confidence at all.

Comment by lebovic 6 days ago

It seems reasonable for a company to require KYC for a product that's dual use – especially a novel one that's built for security research.

Privacy concerns aside, the KYC process for OpenAI was self-serve and took about a minute.

Comment by jiggawatts 6 days ago

Remember the argument that the bad guys using AI to hack systems won't be a problem because all the "good guys" will have access too and can secure their software?

Pepperidge Farm remembers.

Comment by alephnerd 6 days ago

> OpenAI's shit was nearly worthless for cybersec for what, a year already

Plenty of AI for Cybersecurity companies use a mixture of models depending on iteration and testing, including OpenAI's.