AISLE’s autonomous analyzer found all CVEs in the January OpenSSL release

Posted by mmsc 1 day ago

Counter193Comment129OpenOriginal

Comments

Comment by pizlonator 1 day ago

Impressive.

I checked the stack overflow that was marked High, and Fil-C prevents that one.

One of the out-of-bounds writes is also definitely prevented.

It's not clear if Fil-C protects you against all of the others (Fil-C won't prevent denial of service, and that's what some of these are; Fil-C also won't help you if you accidentally didn't encrypt something, which is what another one of these bugs is about).

The one about forgetting to encrypt some bytes is marked Low Severity because it's an API that they say you're unlikely to use. Seems kinda believable but also ....... terrifying? What if someone is calling the AESNI codepath directly for reasons?

Here's the data about that one:

"Issue summary: When using the low-level OCB API directly with AES-NI or other hardware-accelerated code paths, inputs whose length is not a multiple of 16 bytes can leave the final partial block unencrypted and unauthenticated.

Impact summary: The trailing 1-15 bytes of a message may be exposed in cleartext on encryption and are not covered by the authentication tag, allowing an attacker to read or tamper with those bytes without detection."

Comment by arcfour 1 day ago

It would be very surprising to see someone use OCB when GCM exists and is what everyone uses.

Although I agree in principle it is quite scary!

Comment by SAI_Peregrinus 1 day ago

OCB can be a bit faster than GCM, the only reason GCM took over is because OCB was patented. That patent has now lapsed, but since everyone uses GCM the performance advantage of OCB isn't likely worth switching for. Especially since GCM has hardware acceleration, and IIRC OCB can't benefit from that so it may actually decrease performance on modern CPUs.

Comment by arcfour 15 hours ago

IIRC GCM offers additional authenticated data whereas OCB doesn't (or you would have to roll it yourself), right? That would be another reason to pick GCM over OCB.

Comment by pizlonator 1 day ago

> It would be very surprising to see someone use OCB when GCM exists and is what everyone uses.

That is reassuring

Comment by martinald 1 day ago

This really is quite scary.

I suspect this year we are going to see a _lot_ more of this.

While it's good these bugs are being found and closed, the problem is two fold

1) It takes time to get the patches through distribution 2) the vast majority of projects are not well equipped to handle complex security bugs in a "reasonable" time frame.

2 is a killer. There's so much abandonware out there, either as full apps/servers or libraries. These can't ever really be patched. Previously these weren't really worth spending effort on - might have a few thousand targets of questionable value.

Now you can spin up potentially thousands of exploits against thousands of long tail services. In aggregate this is millions of targets.

And even if this case didn't exist it's going to be difficult to patch systems quickly enough. Imagine an adversary that can drip feed zero days against targets.

Not really sure how this can be solved. I guess you'd hope that the good guys can do some sort of mega patch against software quicker than bad actors.

But really as the npm debacle showed the industry is not in a good place when it comes to timely secure software delivery even without millions of potential new zero days flying around.

Comment by teiferer 1 day ago

> the problem is two fold

No, the biggest problem at the root of all this is complexity. OpenSSL is a garbled mess. No matter AI or not, such software should not be the security backbone of the internet.

People writing and maintaining software need to optimize for simplicity, readibility, maintainability. Whether they use an LLM to achieve that is seconday. The humans in the loop must understand what's going on.

Comment by dust42 1 day ago

> People writing and maintaining software need to optimize for simplicity, readibility, maintainability. Whether they use an LLM to achieve that is seconday. The humans in the loop must understand what's going on.

In a perfect world that is.

Comment by MBCook 1 day ago

There’s a reason multiple projects popped up to replace OpenSSL after Heartbleed was discovered.

Let’s see them to do this on projects with a better historical track record.

Comment by PunchyHamster 1 day ago

> 2 is a killer. There's so much abandonware out there, either as full apps/servers or libraries. These can't ever really be patched. Previously these weren't really worth spending effort on - might have a few thousand targets of questionable value.

It's worse than that. In before, operator of a system could upgrade distro's openssl version, restart service and it was pretty much done. Even if it was 3rd party vendor app at the very least you can provide security updates for the shared libs

Nowadays, where everything runs containers, you now have to make sure every single vendor you take containers from did that update

Comment by pjmlp 1 day ago

It would help if regular userspace software wasn't written in languages that were primarly designed to write portable OS kernels.

Even if not all logic errors can be prevented, some of them keep happening by using the wrong tools.

Comment by semiquaver 1 day ago

The silver lining is that if adversarial AI can easily find vulnerabilities, beneficial AI should also be able to find and fix a similar number.

Comment by CharlesW 1 day ago

It's good these bugs are being found and closed. The problems have nothing to do with AI, unless I'm missing something.

Comment by xboxnolifes 1 day ago

If people can use AI to find bugs to close them, people can use AI to find bugs to exploit them. The scale has changed.

Comment by semiquaver 1 day ago

And the project maintainers or their allies can use AI to find bugs and fix them.

Comment by AlotOfReading 1 day ago

The people developing exploits have an obvious way to recoup their token investment. How do the open source maintainers recoup their costs? There's a huge disparity here.

Comment by c-hendricks 1 day ago

Picture the traumatized Mr. Incredible meme with the text "lowering the barrier means more exploits are found"

Comment by charcircuit 1 day ago

>Not really sure how this can be solved.

AI can automatically handle security reports.

Comment by martinald 1 day ago

Of course but how do you distribute the patches? My point isnt that AI can't solve it, but if the project is abandoned then there is no way to get the patches to users.

And even if there is there is an inherent lag. Take these openssl vulns. It's going to go from openssl to (say) Ubuntu. They have to backport the fixes. This isn't trivial as it needs tested and applied to old code versions. These fixes then need applied, and there's no doubt a lot of users not on a "supported" version who won't get the fix.

Even worse something like openssl is almost certainly widely statically linked in many apps/servers. This then requires them to pull it from upstream and repackage, and users to deploy the fix.

So it's a real issue. I'd argue that the industry isn't really able to do this well currently, nevermind if suddenly 1000x the patch frequency happens.

Comment by blibble 1 day ago

> Finding a genuine security flaw in OpenSSL is extraordinarily difficult.

history suggests otherwise

> The fact that 12 previously unknown vulnerabilities could still be found there, including issues dating back to 1998, suggests that manual review faces significant limits, even in mature, heavily audited codebases.

no, the code is simply beyond horrible to read, not to mention diabolically bad

if you've never tried it, have a go, but bring plenty of eyebleach

Comment by timschmidt 1 day ago

The badness cannot be overstated. "Hostile codebase" would be an appropriate label. Much more information available in Giovani Bechis's presentation: https://www.slideshare.net/slideshow/libressl/42162879

If someone meant to engineer a codebase to hide subtle bugs which might be remotely exploitable, leak state, behave unexpectedly at runtime, or all of the above, the code would look like this.

Comment by Swizec 1 day ago

> If someone meant to engineer a codebase to hide subtle bugs which might be remotely exploitable, leak state, behave unexpectedly at runtime, or all of the above, the code would look like this.

I wonder who could possibly be incentivized to make the cryptography package used by most of the worlds computers and communications networks full of subtly exploitable hard to find bugs. Surely everyone would want such a key piece of technology to be air tight and easy to debug

But also: surely a technology developed in a highly adversarial environment would be easy to maintain and keep understandable. You definitely would have no reason to play whackamole with random stuff as it arises

Comment by spease 1 day ago

> Surely everyone would want such a key piece of technology to be air tight and easy to debug

1. Tragedy of the Commons (https://en.wikipedia.org/wiki/Tragedy_of_the_commons) / Bystander Effect (https://en.wikipedia.org/wiki/Bystander_effect)

2. In practice, the risk of introducing a breakage probably makes upstream averse to refactoring for aesthetics alone; you’d need to prove that there’s a functional bug. But of course, you’re less likely to notice a functional bug if the aesthetic is so bad you can’t follow the code. And when people need a new feature, that will get shoehorned in while changing as little code as possible, because nobody fully understands why everything is there. Especially when execution speed is a potential attack vector.

So maybe shades of the trolley problem too - people would rather passively let multiple bugs exist, than be actively responsible for introducing one.

Comment by bsimpson 1 day ago

I wonder what adoption would actually look like.

It reminds me of Google Dart, which was originally pitched as an alternate language that enabled web programming in the style Google likes (strong types etc.). There was a loud cry of scope creep from implementors and undo market influence in places like Hacker News. It was so poorly received that Google rescinded the proposal to make it a peer language to JavaScript.

Granted, the interests point in different directions for security software v.s. a mainstream platform. Still, audiences are quick to question the motives of companies that have the scale to invest in something like making a net-new security runtime.

Comment by khafra 1 day ago

> undo market influence

Pointless nitpick, but you want "undue market influence." "Undo market influence" is what the FTC orders when they decide there's monopolistic practices going on.

Comment by dalmo3 1 day ago

Not pointless. I had no idea what the original wording meant.

Comment by 22 hours ago

Comment by pryce 1 day ago

> Surely everyone would want such a key piece of technology to be air tight and easy to debug

The incentives of different parties / actors are different. 'Everyone' necessarily comprises an extremely broad category, and we should only invoke that category with care.

I could claim "Everyone" wants banks to be secure - and you would be correct to reject that claim. Note that if the actual sense of the term in that sentence is really "almost everyone, but definitely not everyone", then threat landscape is entirely different.

Comment by hdjrudni 1 day ago

I read that whole paragraph with a tinge of sarcasm. There's bad actors out there that want to exploit these security vulnerabilities for personal gain and then there's nation-state actors that just want to spy on everyone.

Comment by otabdeveloper4 1 day ago

> highly adversarial environment

Except it's not. Literally nobody ever in history had their credit card number stolen because of SSL implementation issues. It's security theater.

Comment by rakel_rakel 1 day ago

Another great example from tedunangst's excellent presentation "LibreSSL more than 30 days later".

https://youtu.be/WFMYeMNCcSY&t=1024

Teaser: "It's like throw a rock, you're gonna hit something... I pointed people in the wrong direction, and they still found a bug".

Comment by noname120 1 day ago

I expected much worse to be honest. Vim’s inline #ifdef hell is on a whole other level. Look at this nightmare to convince yourself: https://geoff.greer.fm/vim/#realwaitforchar

Comment by timschmidt 1 day ago

That's a lot of ifdefs, sure. But at least Vim doesn't have it's own malloc which never frees and can be dynamically replaced at runtime and occasionally logs sensitive information.

Comment by consp 1 day ago

As long as you don't statically link you can easily replace malloc (LD_PRELOAD). Many debug libraries do. Why is this so special in openssl? (I don't know if there is some special reason, though openssl is a weird one to begin with)

Comment by timschmidt 1 day ago

Using OpenSSL's malloc may bypass protections of hardened libc mallocs like OpenBSD's.

If memory crosses the boundary between OpenSSL and your app, or some other library, freeing it with a different allocator than the one it was allocated with is undefined behavior.

OpenSSL's allocator doesn't free in in the same ways other mallocs do, which prevents memory sanitization tools like valgrind from finding memory bugs.

OpenSSL has a completely separate idea of a secure heap, with it's own additional malloc implementation, which can lead to state leakage or other issues if not used perfectly at the (non-existent because the entire library surface is exposed) security boundary and is accidentally intermingled with calls to the (insecure?) malloc.

It's just a big can of security worms which may have been useful on odd platforms like VMS, though that's questionable, and only serves to add additional layers of inscrutability and obfuscation to an already messy codebase today. It's not enough to know what malloc does, one must familiarize themselves with all the quirks of both(!) of OpenSSL's custom implementations, which are used precisely nowhere else, to judge the security or code correctness implications of virtually anything in the codebase. There's no good reason for it.

Comment by oefrha 1 day ago

See also The State of OpenSSL for pyca/cryptography

https://cryptography.io/en/latest/statements/state-of-openss...

Recently discussed: https://news.ycombinator.com/item?id=46624352

> Finally, taking an OpenSSL public API and attempting to trace the implementation to see how it is implemented has become an exercise in self-flagellation. Being able to read the source to understand how something works is important both as part of self-improvement in software engineering, but also because as sophisticated consumers there are inevitably things about how an implementation works that aren’t documented, and reading the source gives you ground truth. The number of indirect calls, optional paths, #ifdef, and other obstacles to comprehension is astounding. We cannot overstate the extent to which just reading the OpenSSL source code has become miserable — in a way that both wasn’t true previously, and isn’t true in LibreSSL, BoringSSL, or AWS-LC.

Also,

> OpenSSL’s CI is exceptionally flaky, and the OpenSSL project has grown to tolerate this flakiness, which masks serious bugs. OpenSSL 3.0.4 contained a critical buffer overflow in the RSA implementation on AVX-512-capable CPUs. This bug was actually caught by CI — but because the crash only occurred when the CI runner happened to have an AVX-512 CPU (not all did), the failures were apparently dismissed as flakiness. Three years later, the project still merges code with failing tests: the day we prepared our conference slides, five of ten recent commits had failing CI checks, and the day before we delivered the talk, every single commit had failing cross-compilation builds.

Even bugs caught by CI get ignored and end up in releases.

Comment by bulbar 1 day ago

Wow, that is just crazy. You should investigate when developing software, but for something like OpenSSL... Makes me think this must be a heaven for state actors.

Comment by PunchyHamster 1 day ago

We really need as an industry to move away entirely from this cursed project

Comment by literalAardvark 23 hours ago

I'm surprised AI was even able to find bugs in that.

Given that it's been trained on "regular" code and that presentation points out that openssl might as well be written in brainfuck it shocks me that AI would be able to wrap its pretty digital head around it

Comment by PunchyHamster 1 day ago

> If someone meant to engineer a codebase to hide subtle bugs which might be remotely exploitable, leak state, behave unexpectedly at runtime, or all of the above, the code would look like this.

I'd wager if someone did that the codebase would look better than OpenSSLs

The codebase designed to hide bug would look just good enough that rewriting it doesn't seem worth it.

OpenSSL is so bad that looking at it there is just desire to rip parts straight out and replace them, and frankly only fear-mongering around writing security code kept people from doing just that and only after heartbleed the forks started to try. And that would also get rid of any hidden exploit.

Comment by UltraSane 1 day ago

There is a reason AWS created their own TLS library.

Comment by lumost 1 day ago

It really is just a collection of several dozen research grade implementations for algorithms + a small handful of load bearing algorithms for the entire internet. Surprisingly, OpenSSL isn't the only critical piece of internet architecture like this.

Comment by ryandvm 1 day ago

The longer I develop software, the more I realize just how awful most software engineering it.

Comment by Quarrelsome 1 day ago

maybe this is what blindsides most developers into disregarding the threat of AI to their jobs. We work off some idealised version of what the industry actually is which we presume AI will fail at, instead of the reality.

I remain surprised at how long people can flog horses I figured would be dead decades earlier in enterprise. Too scared to fix fundamental issues and still running off the fumes of vendor lock-in with exasperated end users.

Comment by devsda 1 day ago

Converse is also possible ?

Even with all the best practices, patterns and reviews in place software products often turns out to be held up by hacks and patches.

Add AI and inexperienced developers into the mix, the risk of fragile software increases ?

Comment by Quarrelsome 1 day ago

I worry that software and the industry is more resistent then we might imagine. Consider the insanity of Elon Musk's arbitrary cuts to twitter and the resilience of that platform in the years that followed.

It might simply be the case that buying more tokens and kicking the code enough times might give a "good enough" result for the industry to continue. I don't want to believe this but the discussion of how awful the openssl code base is seems to suggest that might be the case. You just need to automate the process of caution we have around it. We should all be hoping that Gastown fails but I feel like it might succeed.

Comment by bulbar 1 day ago

This case study makes me even think that AI will turn out to be a net positive for overall code quality.

Comment by thaumasiotes 1 day ago

> Consider the insanity of Elon Musk's arbitrary cuts to twitter and the resilience of that platform in the years that followed.

Given the resilience, how can the cuts have been "insanity"?

Comment by Quarrelsome 1 day ago

The insanity is how he enacted them. Like the idea that everyone should come to his office with print outs of the code they've written, or that everyone has to come into HQ to do some all-nighters. Just an absurd hunger-games attitude to his workforce, full of horrific coginative biases and discrimination against some of the workforce (e.g. against those with young children or those with disabilities who might be less able to commit to all-nighters).

Comment by bsimpson 1 day ago

There was an article on here 15ish years ago to the effect of "everything's broken all the time. Everyone who writes software knows it, yet we all tolerate it."

I'd love to find that sometime. Maybe it's time to ask Gemini once again to look for me.

Comment by gostsamo 1 day ago

Is this what you are looking for?

https://www.stilldrinking.org/programming-sucks

Comment by bsimpson 1 day ago

Google guessed this one:

https://medium.com/message/everything-is-broken-81e5f33a24e1

With yours as candidate #2. It's too late here now for me to read them both, but I'll try to go back and check when I have time.

Comment by spease 1 day ago

“…just think, Wally, everything that makes this thing go was supplied by the lowest bidder.”

- astronaut

Comment by dwattttt 1 day ago

Referencing the classic https://xkcd.com/2030

"I don't quite know how to put this, but our entire field is bad at what we do, and if you rely on us everyone will die"

"They say they've fixed it with something called <del>blockchain</del> AI"

"Bury it in the desert. Wear gloves"

Comment by doodlesdev 1 day ago

Honestly, this is absurdly funny, but it makes me wonder whether we'll ever see Computer Science and Computer Engineering as seriously as other branches of STEM. I've been debating recently whether I should keep working in this field, after years of repeatedly seeing incompetence and complacency create disastrous effects in the real world.

Oftentimes, I wonder if the world wouldn't be a bit better without the last 10 or 15 years of computer technology.

Comment by vsgherzi 1 day ago

This is really something that’s making me quite fed up with industry. I’m looking towards embedded and firmware in hopes that the lower in the stack I go the more people care about these type of things out of business necessity. But even then I’m unsure I’ll find the rigor I’m looking for

Comment by eloisius 1 day ago

I’ve been thinking the same thing lately. It’s hard to tell if I’m just old and want everyone off my lawn, but I really feel like IT is a dead end lately. “Vintage” electronics are often nicer to use than modern equivalents. Like dials and buttons vs touch screens. Most of my electronics that have LCDs feel snappy and you sort of forget that you’re using them and just do what you were trying to do. I’m not necessarily a Luddite. I know tech _could_ be better theoretically but it’s distressing to know that it’s also not possible for things to be different for some other reasons. Economically, culturally? I don’t know.

Comment by dumpsterdiver 1 day ago

> makes me wonder whether we'll ever see Computer Science and Computer Engineering as seriously as other branches of STEM

It's about as serious as a heart attack at this point...

Comment by vlovich123 1 day ago

Is it still a critical piece? I thought most everyone migrated to libressl or boringssl after the heartbleed fiasco and serious people took a look at OpenSSL and started to understand the horror show that is the codebase and also development practices that clearly have not gotten better, if not gotten even worse.

Comment by cryptonector 1 day ago

> history suggests otherwise

The methodology for developing and maintaining codebases like OpenSSL has changed!

> no, the code is simply beyond horrible to read, not to mention diabolically bad

OpenSSL? Parts of it definitely are, yes. It's better since they re-styled it. The old SSLeay code was truly truly awful.

Comment by fulafel 1 day ago

We don't know how to secure C codebases by manual review. It's been well known to security engineering people for decades. And has been wider industry and academic consensus for a long time. It's like "is man-made climate change real".

(We don't know how to secure other codebases either, but C is harder since its memory safety story is like a chainsaw juggling act so code has classes of vulnerabilities that other languages don't and this eats a lot of the attention).

Comment by rzerowan 1 day ago

Also werent a lot of deadend code removed and vulns patched into what would become LibreSSL.

Would be interesting to see if any of those found exist there.

Comment by nextaccountic 1 day ago

Why do people use OpenSSL? Or any other library that forked from it

Why not start from a clean slate? Companies like Google could afford it

Comment by josefx 1 day ago

Security certifications are one reason. OpenSSL maintains a module for FIPS compliance, which includes an entire boatload of weak and broken algorithms nobody else bothers with.

Comment by nextaccountic 1 day ago

This kind of security certification seems like the exact opposite of actual security

Comment by PunchyHamster 1 day ago

It is. There are other related issues like at some point RedHat patched back options removed/changed in openSSH 7.0 because

* they upgraded a major release (6.x to 7.x) in "stable" channel of their distro * their customers ran some ancient stuff that required those options.

We've failed a security audit because our checks just compared OpenSSH version ("if version is above this it doesn't need any change in config") while Red Hat's OpenSSH version was downgraded to earlier version settings/security issues

Comment by sharms 1 day ago

AWS actually has two libraries they use instead: s2n and aws-lc https://github.com/aws/s2n-tls https://github.com/aws/aws-lc

Comment by lmm 1 day ago

Because as horrible as the OpenSSL code is, the best available clean implementation would mean using a language that's weird and French.

Comment by nextaccountic 1 day ago

Do you mean HACL* / ValeCrypt / EverCrypt?

A number of projects like Firefox and the Linux kernel uses them. It's boring at that point. The generated code is C and assembly can be used like any library, but it has been formally verified.

But, there is ring and rustls too. A number of projects are shifting to it

Comment by lmm 1 day ago

Hah, I meant ocaml-tls which I think is still the most mature option in this space. But yeah there are other approaches.

Comment by snvzz 1 day ago

Instead of everybody switching to LibreSSL, we had the Linux Foundation reward OpenSSL's incompetence with funding.

We are still suffering from that mistake, and LibreSSL is well-maintained and easier to migrate to than it ever was.

What the hell are we waiting for?

Is nobody at Debian, Fedora or Ubuntu able to step forward and set the direction?

Comment by lovich 1 day ago

I can read C/C++ code about as well as I can read German. Bits and pieces make sense but I definitely don’t get the subtleties.

What’s eye bleachy about this beyond regular C/C++?

For context I’m fluent in C#/javascript/ruby and generally understand structs and pointers although not confident in writing performant code with them.

Comment by jeffbee 1 day ago

For one thing, "C/C++" is not a thing. If you see C-like C++, that is C.

Part of OpenSSL's incomprehensibility is that it is not C++ and therefore lacks automatic memory management. Because it doesn't have built-in allocation and initialization, it is filled with BLAH_grunk_new and QVQ_hurrr_init. "new" and "init" semantics vary between modules because it's all ad hoc. Sometimes callees deallocate their arguments.

The only reason it needs module prefixes like BLAH and QVQ and DERP is that again it is not C++ and lacks namespaces. To readers, this is just visual noise. Sometimes a function has the same name with a different module, and compatible function signature, so it's possible to accidentally call the wrong one.

Comment by assanineass 1 day ago

[dead]

Comment by hnmullany2 1 day ago

[dead]

Comment by dnw 1 day ago

"We submitted detailed technical reports through their coordinated security reporting process, including complete reproduction steps, root cause analysis, and concrete patch proposals. In each case, our proposed fixes either informed or were directly adopted by the OpenSSL team."

This sounds like a great approach. Kudos!

Comment by mvkel 1 day ago

So here we have OpenSSL, coded by humans, universally adopted by the Internet, universally deemed to be terrible code.

More evidence that "coding elegance" is irrelevant to a product's success, which bodes well for AI generated code.

Comment by not_a_bot_4sho 1 day ago

If anything, this is evidence that coding elegance has value.

The unexpected part here being that AI brings specks of elegance to a terrible, inelegant codebase.

Comment by lmm 1 day ago

The sad reality is that if your code is available for free and works most of the time, nothing else matters. I'm not sure I would call it "product success" given that OpenSSL's income is enough to cover, like, one dude in a LCOL country some of the time.

Comment by SkiFire13 1 day ago

I find it scary TBH that we're on track to have more OpenSSL-level software.

Comment by chris_wot 1 day ago

It seems to me that after seeing some of the presentations by the LibreSSL folks that OpenSSL is not evidence of elegant code.

Comment by kajaktum 1 day ago

Openssl? Code elegance?

Comment by jkingsman 1 day ago

I think they're saying that OpenSSL is NOT elegant, but that it is successful regardless; hence, code elegance is irrelevant to whether a product is successful or not (and thus that horribly ugly LLM-generated code has a shot at becoming successful).

Comment by 1 day ago

Comment by hnmullany2 1 day ago

[dead]

Comment by ggm 1 day ago

OpenSSL is a very odd codebase, it's grown by accretion, under many stewards, with several flavours of coding belief, over time from SSLEAY which Eric Young coded over 2 decades ago. It had chip-specific speedups from the days of the Intel 486.

I was part of a body which funded work to include some stuff in the code, and the way you take something like X509 and incorperate a new ASN.1 structure inside the code, to be validated against conformance requirements (so not just signing blindly over the bitstream, but understanding the ASN.1 and validating it has certain properties about what it says, like not overlapping assertions of numeric ranges encoded in it) is to invoke callouts from deep down, to perform tasks and then return state. You basically seem to have to do about a 5 layer deep callout and return. It's a massive wedding cake of dependency on itself, it personifies the xkcd diagram of "...depends on <small thing>" risks.

I'm not surprised people continue to find flaws. I would like to understand if this approach also found flaws in e.g. libsodium or other more modern crytography, or in the OpenBSD maintained libreSSL code (or whatever it is) or Peter Gutmann's code.

OpenSSL is a large target.

Comment by tqk_x 1 day ago

Unfortunately you have to "request a demo" while supplying a company name and getting past a Cloudflare CAPTCHA.

So again this is not reproducible and everything is hidden behind an SaaS platform. That is apparently the future people want.

Comment by bandrami 1 day ago

I'm bearish on AI creating working software but bullish on AI doing this kind of thing

Comment by cryptonector 1 day ago

AI is definitely creating working software.

It's also leading people to submit hallucinations as security vulns in open source. I've had to deal with some of them.

Comment by lmm 1 day ago

Define "working". It's creating software that does what someone wanted some of the time, sure.

Comment by teiferer 1 day ago

Look around you. Our industry has cultivated that this kind of software is everywhere.

Comment by bandrami 1 day ago

It's... really just not, though

Comment by teiferer 1 day ago

There are isolated islands of reliable, high quality, low bug, well maintained software. The rest is crap.

Comment by bandrami 1 day ago

Nah, if that were true there would be a lot more new software available. What's it working at is making developers feel busy, which is itself a worthwhile task.

Comment by rascul 1 day ago

Maybe it's available but you don't realize it.

Comment by bandrami 1 day ago

Maybe. Sounds like a real marketing failure in that case

Comment by dboreham 1 day ago

There may be some other limiting factors on the rate of generation of new useful software. E.g. money to market it, humans to specify it, perhaps there's enough software already...

Comment by bandrami 1 day ago

If there's enough software already than LLMs are the worst-timed invention in human history...

Comment by soulofmischief 1 day ago

Are you unable to create working software with frontier models?

Comment by bandrami 1 day ago

I am, though it always takes me longer than just writing it myself because I have to fix so much (which may be a function of the kind of software I write). But more importantly the development team I support as an admin absolutely loves their agents or whatever they're being called this week and yet isn't giving me stuff that I can move into operations any faster than they were a year ago.

Comment by soulofmischief 1 day ago

What industry, and what kind of software? Your assessment was generally spot on a year ago, but things have changed dramatically in the last quarter, so I'm curious how fresh this assessment is.

Comment by bandrami 1 day ago

Well, multiple types of software. The avionics simulation software we make isn't really a candidate for AI both because of procurement requirements and the fact that as of this month (we do check periodically) no LLM really gets how to do realtime Linux processes (this may be downstream of the fact that most writing about this topic on the Web is catastrophically wrong). The stuff we can use AI on is just generic customer-facing web schlock and it sure looks like we're trading dev time for integration time for what ends up being a wash.

Comment by soulofmischief 1 day ago

Interesting, thanks for sharing!

As for the web stuff, these tools are great in the hands of thoughtful, attentive, experienced engineers who have developed the muscles for knowing how to slap these models into shape. For anyone else, I agree right now that they can be more headache than they are worth.

I get a lot of velocity out of Opus 4.5 and spend 8-20 hours a day coding with it nearly every day, but I am constantly, multiple times an hour, screaming and yelling at these things, getting frustrated and bewildered by their output, etc. It is absolutely a tradeoff, but thankfully the tradeoff for me is frustration and mental energy, instead of correctness or performance. But left alone, these models drive in circles and tear everything up along the way.

I totally believe you about these models having difficulty with realtime programming. It's a more niche field with less example training material. Out or pure curiosity I do wish I was able to see exactly where the failure modes arise. I wonder how things will be at the end of 2026, because 2025 was a game changer for many domains.

Comment by what 1 day ago

Why don’t you post some of the software the AI has produced for you in the last quarter? Shouldn’t be a problem since it works so well now.

Comment by soulofmischief 1 day ago

To what, prove something to you? I have nothing to prove and feel no obligation to do so.

If you want to understand what current technology is capable of but can't ask without attitude, you can look it up yourself.

Comment by yes_man 1 day ago

I don’t want to discredit the authors but just want to offer couple of hypothetical points in these paranoid times.

From a marketing angle, for a startup whose product is an AI security tool, buying zero-days from black market and claiming the AI tool found them might be good ROI. After all this is making waves.

Or, could it be possible the training set contains zero-day vulnerabilities known to three-letter agencies and other threat actors but not to public?

These two are not mutually exclusive either. You could buy exploits and put them in the training set.

I would not be surprised if it is legit though.

Comment by mnicky 1 day ago

To your second point - why would you need this? There are _plenty_ of previously found CVEs to train on.

Also, I don't think the three letter agencies would share one of the most prized assets they have...

Comment by aster0id 1 day ago

How many false positives did the AI throw up?

Comment by b1temy 1 day ago

Even if it does have false positives, I expect it would make a nicer starting point for finding and verifying bugs/vulnerabilities, compared to wading through the entire codebase until you find something. Even if it is a false positive, it would probably be due to sketchy looking code (hopefully, unless it hallucinated completely new code) that you can take a look at, and maybe spot something else that the AI didn't catch.

Besides the HN submission, XBOW and Hacktron AI has found plenty of vulnerabilities in code.

Comment by tyre 1 day ago

Does it matter? They found 12 vulnerabilities. Clearly there was enough signal:noise that they could uncover these as real.

It doesn't look like they had 1 AI run for 20 minutes and then 30 humans sift through for weeks.

Comment by microtonal 1 day ago

Does it matter?

Yes, we have been on the receiving end of AI generated bug reports and in the vast majority of cases they are really bad. But you still need humans to sift through them. And when you ask the submitter questions, it’s often clear that they just give the questions to an LLM again to answer.

It costs a huge amount of human manpower, so if the company who made this had an AI based solution with a far lower false-positive rate, that would be great.

Comment by awesome_dude 1 day ago

> It doesn't look like they had 1 AI run for 20 minutes and then 30 humans sift through for weeks.

It does, though, look like they were running their AI over the codebase for an extended period of time (not per run, but multiple runs over the period of a year)

> Does it matter?

Hell yes, false reports are the bane of the bug bounty industry.

Comment by awesome_dude 1 day ago

They don't appear to go into detail about anything except how great it is that they found the bugs, what those bugs were, and how rare it is for other people to find bugs.

I think that it would be helpful from a research point of view to know what sort of noise their AI tool is generating, but, because they appear to be trying to sell the service, they don't want you to know how many dev months you will lose chasing issues that amount to nothing.

Comment by ape4 1 day ago

I wonder too. Did it take many human hours to verify everything?

Comment by crm9125 1 day ago

"Humans + AI" ...

Without Humans, AI does nothing. Currently, at least.

Comment by adzm 1 day ago

Just wait until AI has its own money

Comment by belter 1 day ago

Wait until AI starts using AI

Comment by baby 1 day ago

This sounds amazing but not too much info on how it worked

Comment by _JoRo 1 day ago

Does anyone have any recommendations on best practice security methods? As others have said, it sounds like there may be an order of magnitude more vulnerabilities found / exploited, and I'm wondering if security such as 2FA and Password Managers will be enough? Should people be getting on board with other protections such as security keys?

Comment by jeffbee 1 day ago

I don't know why you're still using OpenSSL but if you're able to switch I note that BoringSSL was not affected by any of the January 2026 OpenSSL advisories, and was also not affected by any of the advisories from 2025, and was affected by only one of the 2024 advisories. I also note that I don't see any hasty commit activity to s2n-tls that looks like a response to these advisories.

Better software is out there.

Comment by cookiengineer 1 day ago

I wanted to mention WolfSSL.

I like to recommend that project because it has a very transparent vulnerabilities approach, and is in my opinion written a lot more sane than OpenSSL which is somewhat not using standard C features because it always implements everything from scratch like a kernel does.

But yeah, anyways, WolfSSL comes from the embedded area in case that's your thing.

[1] https://www.wolfssl.com/

[2] https://github.com/wolfssl/wolfssl

Comment by Thaxll 1 day ago

What's the kind of prompt / flow to get Claude to work on those security tasks?

Comment by viraptor 1 day ago

Same as for people. You establish what the threat model is and then have multiple approaches. For example going through all interesting operations, tracking down their inputs and data flow, then looking for edge cases along the way. If you have enough time / tokens, this becomes more of a spreadsheet/checklist exercise. The more experience you have, the better you can prioritise that list towards paths that are more likely to be disrupted.

Comment by panzi 1 day ago

What kind of AI does this use?

Comment by mrbluecoat 1 day ago

Like any powerful tool, used responsibly in the right hands it could lead to great good; in the wrong hands or used irresponsibly, it could be extremely dangerous.

Comment by move-on-by 1 day ago

Pretty impressive. Whether you think AI is a bubble or not, we all benefit from these findings.

As for all the slop the Curl team has been putting up with, I suppose a fool with a tool is still a fool.

Comment by apexalpha 1 day ago

The author of cURL posted on LinkedIn about this with praise and the addition that AISLE has reported vulns to them in the past.

https://www.linkedin.com/posts/danielstenberg_vulnerabilitie...

Comment by wolfi1 1 day ago

ok, so the USP for this analyzer is: 'He hackers, if you look for zero-days we've got the tool for you!'

Comment by portender 1 day ago

The fun thing to me here is that a ton of really creative thinkers are going to have access to tools (LLM agents) that allow them to test their thinking quickly. I dearly hope that this leads to a prolonged phase of pain and loss.

We made good choices when we decided the information on the internet should be delivered by simple, open protocols.

We made bad choices when we decided that the information on the internet didn't need to be verified, or verifiable.

Then we slipped on our good choices, because our bad choices let robber barons claim the verified or verifiable case.

And then we were left an explosive entropy shit-pile.

But now the new tools the new overlords are paying us to use will help us break free from their shackles, bwahahahahahahahahahahahah!!!!

Comment by ChrisArchitect 1 day ago

Related:

AI discovers 12 of 12 OpenSSL zero-days (while curl cancelled its bug bounty)

https://www.lesswrong.com/posts/7aJwgbMEiKq5egQbd/ai-found-1...

Comment by jibal 1 day ago

The title change from "AISLE" to "AI" is misleading. As the article states,

> This doesn't mean that AI can replace human expertise. The OpenSSL maintainers' deep knowledge of the codebase was essential for validating findings and developing robust fixes. But it does change the SLA of security. When autonomous discovery is paired with responsible disclosure, it collapses the time-to-remediation for the entire ecosystem.

Comment by rascul 1 day ago

Only 12?

Comment by ChrisArchitect 1 day ago

Related:

OpenSSL: Stack buffer overflow in CMS AuthEnvelopedData parsing

https://news.ycombinator.com/item?id=46782662

Comment by TalkWithAI 1 day ago

[dead]

Comment by ktimespi 1 day ago

Link seems to be down... But also, considering curl recently shut down its bug bounty program due to AI spam, this doesn't really inspire much confidence.

Comment by M0dEx 1 day ago

[dead]