Do the math: How we took over ChatGPT accounts

Posted by armcat 1 day ago

Counter359Comment176OpenOriginal

Comments

Comment by yellow_lead 1 day ago

> OpenAI confirmed both frontends rejected the reported messages, resolved the issue, and awarded a total bounty of $5,000.

It requires the user to click a link. But I still find this bounty very low. Others may not report them if such a small bounty is given.

Comment by Catloafdev 1 day ago

A bounty like this is a polite "thanks asshole, fuck off, don't do it again" rate.

Paying effectively nothing for vulnerabilities just means they don't want to deal with the program.

Wild to think a company supposedly worth a $trillion+ is doing shit like this.

Comment by Forgeties79 1 day ago

Because they shouldn’t be worth that tbh. They are a cash burning operation hoping to get what they can before the music stops.

Comment by andai 1 day ago

They have 1 billion active users.

Comment by mossTechnician 23 hours ago

Do those billion users contribute to a profit or loss?

Comment by somenameforme 1 day ago

I'd love to know the demographic breakdown there. For instance I know students in some relatively isolated corners of the world are still, ubiquitously, using it to cheat. And a quick search gives around 1.4 billion students globally.

Obviously it's not just students, but the fact it [numerically] could be is rather amusing and makes me surprised that it's only 1 billion active users, since GPT has a lock on name recognition amongst low information users.

Comment by raegis 1 day ago

> They have 1 billion active users.

I'm naive, but is this possible? The power required per user to run those GPUs seems like enough to run an air conditioner. For a billion people everyday or so seems like enough to overload power plants.

Comment by elpocko 16 hours ago

> The power required per user to run those GPUs seems like enough to run an air conditioner.

Not true. Maybe you're thinking of computer games? Inference requires much less energy per user session than a session of your average AAA game title. And there are more active gamers at any time than active ChatGPT users.

Comment by Forgeties79 15 hours ago

You’re comparing 1 billion people doing bleeding edge gaming to LLM usage. Feels a bit like your thumb is on the scale here. Even under lower load GPU’s do in fact use a lot of power, as evidence by the massive power buildouts that we are seeing across multiple countries.

Comment by 12 hours ago

Comment by 16bitvoid 1 day ago

And only about 5% (50M, last time it was reported) of them pay. The vast majority of those are using the free plan.

Comment by Forgeties79 17 hours ago

Yes Silicon Valley gives things away to build a high DAU count before the squeeze we’ve seen it for almost 2 decades now. That tells us nothing of the longterm health of their business.

Comment by fsckboy 1 day ago

A company worth trillions does not have trillions, that is the expectation for what it will be worth in the future based on money it has already spent and will need to keep spending.

Moreover, that translates to "it's actually carrying a huge nut* that puts extremely intense pressure on the company to figure out how to make as much money as possible as soon as possible."

(*nut is a slang word from loan sharking that has been adopted by (personal et al) finance people. It means "how much do you owe each month?" in the sense of, if you want to do a startup, advisors could ask, "what's your nut, and how much do you have left to self-invest?" Because monthly expense can be combined to a lump sum, it can also mean the size of the overall loan where the monthly interest is called the vig, "as long youze got da nut, you owe me da vig;" from vigorish, which would also require an etymology, but I'm assuming it's the word vigor combined with licorice meaning bitter tasting medicine, and nut because squirrels busy themselves carrying nuts.)

----

and if we can... and remember flame-prone people, if you reject the "can" premise, then the rest of the argument goes away, and you don't need to dispute it, you just say "you can't nuff said"; people here frequently seem challenged by propositions.

so, if we can...loosely describe the attitude you show here toward high market cap companies to be compatible with "left" or "socialist" or "populist" views shared by many others here, then I'll point out that at the same time wanting higher market prices paid for bounties because 'sploits are so expensive, is actually advocating for the same dread capitalist market based solution that leads to trillion dollar companies that never even had a trillion to begin with.

Comment by ibaikov 1 day ago

I took over user accounts in Microsoft Copilot (there are caveats but still), and they asked me to provide additional information of the exploit (they wanted a video) and before I sent it they fixed the vulnerability, didn't pay me a cent. Fair? Idk.

Comment by pvtmert 1 day ago

Typical $MSFT behavior, I can see why NightmareEclipse (ie. YellowKey zero-day exploit) got released without prior fix.

Comment by Biganon 21 hours ago

We need an (official) third party, like an escrow. You report to them, they manage the talking with the company, they tell the judge you did nothing wrong because you reported to them, THEY use tables and shit to decide that such and such attack vector is worth this percentage of the company's revenue, this team would have received a hefty sum

Comment by zeroq 1 day ago

It's a complicated issue in current LLM world where every day is a zero day, but fuck me sideways, company that claims to be worth trillion of dollars paying 5k for such an exploit is insulting at best.

It's been years since I hanged out with wizards selling such exploits like trading cards, but I bet there are still tons of brokers that will be more than happy to wire you legal money for such finding.

I mean, kudos for the team for doing the right thing. It's great PR and another big one in the portfolio, but, as other have said already, I doubt the next group will even bother contacting OpenAI.

Comment by windexh8er 1 day ago

I find Sam Altman recent comments very "low" as well, neither matters. So in that case I think bounty hunters should just cold shoulder idiocracy and sell these to the underground. Let their customers be abused and complain about how OAI isn't protecting them. Each one of these types of events could cost them thousands of dollars in lost revenue and/or internal support.

Comment by rootnod3 1 day ago

2 trillion $ companies should have the money to pay for these exploits. If they don’t, go tucking exploit them. If they don’t give a fuck about stealing knowledge from copyrighted material and private git repos, light em up.

Comment by throwaway98797 1 day ago

some people don’t want to be criminals as a matter of ethics

but hey you do you

Comment by odyssey7 1 day ago

True, however:

Ethics and legal requirements shouldn’t be conflated.

There are times when ethics are a higher standard than what the law requires, and in that case the ethics are no less important.

Comment by phoghed 1 day ago

Yes, you have a moral obligation to Sam Altman’s trillion dollar business.

Comment by BalinKing 1 day ago

On the contrary, many (including myself) would argue that "the victim's amount of money/power" has absolutely no ethical relevance: "You shall do no injustice in judgment; you shall not be partial to the poor nor defer to the great, but you are to judge your neighbor fairly" (Leviticus 19:15).

Comment by OoooooooO 1 day ago

Citing a fantasy novel to prove your point doesn't add anything to the discussion.

Comment by RHSeeger 1 day ago

But some would argue that reducing the amount of resources and power available to someone who has immense amounts of both and is causing great harm to society as a whole - is ethical.

Comment by bsoles 1 day ago

Citing Leviticus for moral guidance is rich. Should we also kill people having affairs? (Leviticus 20:10).

Comment by BalinKing 1 day ago

I'd argue that the moral principles of the Levitical law (e.g. "adultery is wrong") are not inherently bound to the specific punishments (e.g. "adultery is a capital crime") prescribed for ancient Israel.

Comment by bsoles 19 hours ago

Oh, pick and choose your own crime and punishment interpretation of the Bible. How convenient.

Comment by BalinKing 17 hours ago

To defend against the accusation of cherrypicking: I don't actually think the original punishments were objectively morally wrong (even though they certainly feel harsh to my modern sensibilities)—sin is no joke. At the same time, I also don't believe that the punishments (or even all of the crimes, i.e. the liturgical ones) were intended to apply to all polities throughout history. Admittedly, my argument would be primarily theological in nature[1], so I don't expect it to be convincing... we may have to agree to disagree :-P

[1] Namely, "ancient Israel is the only divinely instituted theocracy in history, so many things just won't generalize to more 'normal' polities". And/or, see Jesus' treatment of the Mosaic Law in the gospels (and the New Testament more generally).

Comment by ajxs 1 day ago

Unfortunately my details (address, bank details, government IDs) have been leaked a handful of times now because companies that I do business with have been hacked. Each time the company involved has faced practically no real penalty aside from a minor hit to their reputation. Your moral obligation isn't to Sam Altman, it's to his customers, who are real people.

Comment by jfyi 1 day ago

No, we have a moral obligation to make sure Sam Altman's trillion dollar business does ethical shit.

If that means taking chunks of it because he can't be a responsible steward, so be it. It would behoove him to get on board before that is necessary.

Comment by codechicago277 1 day ago

This isn’t one of those times.

Comment by whamlastxmas 1 day ago

Only if you don't believe the means justify the ends. Not saying I believe that myself

Comment by tonyedgecombe 1 day ago

Is it criminal to sell an exploit?

Comment by echoangle 1 day ago

Facilitation of a crime is a crime too. So I think in the US and most western countries, yes. At least if it gets used illegally later.

Comment by tadfisher 1 day ago

Pretty sure you need actual knowledge of a planned crime for that to stick. But I'm also sure the defense will be expensive.

Comment by echoangle 1 day ago

I'm not sure but if negligence is punishable too for facilitation, it should stick, right? I think it's reasonable to expect that most buyers of exploits try to use them in illegal ways (or are feds acting like that to catch you).

Comment by cowlevel 1 day ago

They won't have to get you for facilitating the later crime, they can get you for testing the exploit on your own account, which is also a crime since OpenAI's computer system is involved.

Comment by 1 day ago

Comment by TheRoque 1 day ago

I don't think so, businesses like Zerodium bought zero days in plain sight and I don't think they faced legal consequences. They don't operate anymore though. So basically I'd say it depends who you sell it to and how.

Comment by qlte 15 hours ago

It's my understanding that those groups buying exploits in plain view sold exclusively to government agencies which is why their business model was tolerated.

Comment by Black616Angel 1 day ago

Yeah, and some people just want the multi-billion dollar company that is currently destroying the planet while also permanently crippling social connections to just stop.

Comment by 1 day ago

Comment by 1 day ago

Comment by blurbleblurble 1 day ago

If only poverty were so simple

Comment by dbspin 1 day ago

This isn't like stealing from a business. These are individuals most private conversations, potentially including their bank accounts etc. Poverty is no defence - and yes, I've spent a great deal of my life barely scraping by. You'll find poor people are significantly less tolerant of theft, especially from individuals, than rich. Whether that evidences the theft inherent in most forms of wealth generation I'll leave up to the reader.

Comment by windexh8er 1 day ago

Treating people and businesses ethically that don't reciprocate doesn't work, clearly. Those people and businesses cry in their milk while they do exactly the same thing. There are no ethics in play here.

But, hey if you want to defend billionaire narcissists that are destroying our planet and trying to take computing freedoms then I guess: you do you.

Comment by oth001 1 day ago

Ah yes, OpenAI, the poster child for ethics

Comment by krkdkfkfkfk 1 day ago

[dead]

Comment by nullsanity 1 day ago

[dead]

Comment by dgellow 1 day ago

That’s so low. That’s the level of bounty I awarded researchers when I was managing the bug bounty program at a small startup, it’s just ridiculous coming from a large company like OpenAI

Comment by amelius 1 day ago

But on the other hand, you can use AI now to find those vulnerabilities.

Comment by tescreal 1 day ago

If that is so, why doesn't OAI have a swarm doing this around the clock?

Comment by user43928 1 day ago

Surely with all their security programs, they would run testing on the ChatGPT implementation.

I speculate that the AI didn't find this and that human effort and intuition for where to look were needed to find this vulnerability.

Comment by hxugufjfjf 1 day ago

Who says they don't? And if they don't, maybe its because others are using their hard-earned money to buy tokens from them to do it, so they don't really have to.

Comment by Forgeties79 1 day ago

AI evangelists would’ve latched on to it already and would be using it to preach the good word.

Comment by DeepSeaTortoise 1 day ago

Having bit the bullet, I found OAI's models actually quite helpful and most of their offering half-decent. But their whole German website was incredibly concerning and the only reason I went for it was people claiming they're not far behind Anthropic on coding and generally superior for everyday tasks like coming up with recipes to cook from your leftovers.

Like half of the website was dysfunctiona and translations incredibly bad to the point of being nonsensical. That is if you were able to read them at all, because many just went off screen. Like, that's what your evil competitor does to your website after hijacking the domain, not you yourself, ffs

Comment by 8note 1 day ago

thats reasonably the state of anthropics code output as well - not much care and polish

Comment by baxtr 1 day ago

They should have paid in tokens?

Comment by dgellow 1 day ago

A bounty amount is related to the blast radius and overall impact, not the effort required to find the issue

Comment by dist-epoch 1 day ago

This is a pet peeve of tptacek:

> server-side vulnerabilities are worth very little, because they quickly get patched.

Comment by dannyw 1 day ago

The "take over any instagram/meta account through their AI password reset chatbot" was going around and sold on shady forums for ~1 month before being patched.

Comment by throwaw12 1 day ago

then server side vulnerabilities should not be reported, instead sell the data, more profitable

Comment by petcat 1 day ago

Very few people would even have the knowledge of who, how, or where to "sell" such vulnerabilities. Going through a public bounty program is pretty much always preferable.

Most people don't want to be felons.

Comment by pixl97 1 day ago

Don't worry, as more and more people get put of of work by AI the number of people willing to be felons will increase. Hell, they might even become president.

Comment by cowlevel 1 day ago

The president can't be a felon by definition, because it's legal to do absolutely anything if you're the president.

Comment by thaumasiotes 23 hours ago

The legality of most activities doesn't change, but you can't be prosecuted while you're president.

Comment by tadfisher 1 day ago

I am sure this is a crime, but what crime?

Comment by throwaw12 1 day ago

I think people who can hack, also know where they can sell it

Comment by petcat 1 day ago

Anyone can "hack". It's not really a skill anymore.

Comment by qup 1 day ago

The recent South Korean hacker reportedly wasn't a hacker at all, just asked Claude to handle it.

I'm not that kind of hacker, but I guarantee I could be, complete with an abliterated model on my local rig, in a few minutes. Claude would happily set it all up for me, too.

In reality, I suppose I'm about to keep my rig busy staying ahead of the hackers.

Comment by tomjen3 1 day ago

I'm sure there are places that would sue you for that, and it's not worth the bother. Bug bounties really compete mostly with people like me who would say, "Haha, I found an issue with Meta; they're gigantic idiots." And then make an X post or anything about it.

Comment by hashstring 1 day ago

Technically this is what you’d call a client-side vuln, but yes, server-side as in cloud.

Comment by thatsabadlook 1 day ago

Not worth the effort for a company supposedly worth a trillion dollars. The problem is oai can't afford all of its mistakes at the usual rate and they make bad press. A 100k bug could make a headline, but 5k for the same bug requires the average person to be educated enough to understand how bad these issues are.

Comment by varispeed 1 day ago

Should be given $5-10million under NDA. No headlines.

Comment by layer8 1 day ago

The bounty payouts must be public in order to serve as an incentive.

Comment by varispeed 1 day ago

That incentive works both ways. For instance employee sneaking in a security hole and then selling it to a mate so they can collect the bounty.

Comment by 1 day ago

Comment by weird-eye-issue 1 day ago

Just imagine what the press would be when somebody breaks that NDA

Comment by goodmythical 1 day ago

>should be paid multiple years of a high end vuln researcher salary per single discovery

What? Why would anyone work at a company paying them that much less than they pay the contractors?

Comment by darig 1 day ago

[dead]

Comment by kamban 1 day ago

They have just announced free API credits for Pro plan users up to $500 per month. Irony.

Comment by zxilly 1 day ago

To be fair, based on the profit margins indicated in DeepSeek’s reports, a $500 API credit might only cost them $5 to provide.

Comment by jryle70 1 day ago

OpenAI acknowledging and fixing the bug is worth millions

Comment by yeahsure 1 day ago

Yeah... idk, sounds like when I was starting out almost 30 years ago my clients constantly lowballed me saying "but think of the exposition! this project will give you" lol

Comment by jryle70 1 day ago

Were they not right? Just like money giving you some runway but will run out, the exposure is still only a leveler, it's still up to you to take advantage of it.

Also, they were not OpenAI.

Comment by alfiedotwtf 1 day ago

The buyer of your second vulnerability all depends on how you’re treated for your first vulnerability

Comment by varispeed 1 day ago

Indeed, it's like "here peasant, have this fortune of $5,000, may it change your life". Pathetic.

Comment by siva7 1 day ago

that wouldn't be the time worth of many people here to make the security report for OAI..

Comment by cowlevel 1 day ago

50 hours at $100/hour? To write a straightforward vuln report with AI?

Comment by lxgr 1 day ago

Oof. Together with the fact that it's possible to remotely access the file system (!) of computers ever having started a "Work" chat in the ChatGPT desktop app [1], this seems pretty bad.

[1] https://news.ycombinator.com/item?id=50078284

Comment by albert_e 1 day ago

Claude Code terminal has "remote control" option activated via /rc command. -- somewhat similar in that it still runs the session on the PC/Mac but allows the Claude authenticated user to access the session via internet / app

I am guessing this ChatGPT functionality is a similar Codex feature?

But this session is disconnected when the conversation is /exit ed

Comment by lxgr 1 day ago

Sounds similar, but the problem is exactly that I don’t think there’s any way to disconnect such a session, nor to even see that it is connected to my computer in the first place. There’s a small notice when initially starting it on the mac, that’s it.

Comment by randomblock1 1 day ago

On the Work environment selector, "in the cloud" says "available on any device and can access your computer". Which seems pretty clear to me. (not that the labeling makes the security issue any better)

Comment by zahlman 1 day ago

Something about the writing style here makes it hard for me to follow what actually happened. The section about JSONP for example seems to be explaining how JSONP works and how it's designed as an attempt to work around these attacks, but that explanation is then passed off as an explanation of how they bypassed the protection. And the whole premise seems to be about how manipulating the Preview content let them work through multiple defensive layers, but I don't see an explanation of how they directly controlled that content in the first place. I would have thought that the only control you have over the content of "an ordinary public ChatGPT share" is the prompt text, and that any migration of that text into "a small working webpage using its Preview feature" is up to the model deciding to generate that "HTML block" in response.

Comment by 0c3ca83 1 day ago

They're writing for bots, not humans. This is a marketing piece. They're trying to get themselves to come up as the answer when people ask chatbots questions.

You're not the target audience of writing on the Internet anymore.

Comment by cowlevel 1 day ago

It's AI written so of course it's hard to follow.

Comment by morgaesis 1 day ago

Your conclusion does not follow from your premise.

Comment by mediaman 1 day ago

It does if LLMs are generally poor writers, which they currently are, because it is extremely difficult to RLVR essay quality.

Comment by krackers 1 day ago

It's surprising that using JSONP in this way is even allowed. At one point google explicitly made sure to prevent this flavor of attack

https://stackoverflow.com/questions/2669690/why-does-google-...

Comment by kenmacd 1 day ago

hnslop (using salahadawi, using Pangram) puts this at 100% AI written, so that's probably why.

Report: https://www.salahadawi.com/hacker-news-ai-detector/50314699

hnslop: https://hnslop.nilsherzig.com/

Comment by zahlman 1 day ago

Oh, I don't doubt that. But most of the time I can pick out useful information from the claudeslop. This piece seems almost deliberately obscurantist.

Comment by dumberquestions 1 day ago

Does no one write their own blogs anymore?

Comment by trungto 1 day ago

For my personal blog [0], I am still writing without LLM assistance. The goal is to test how well I understand the topic I am writing about, as well as to improve my own writing.

[0] https://trung.to

Comment by zahlman 1 day ago

I write mine, and will continue to do so, by hand. I would never consider letting an LLM touch my writing.

Posting frequency is another matter…

Comment by armcat 1 day ago

I actually still write my own blog [1] :sweat_smile:

Pure html by hand, using vim. No frameworks, no nothing, single stylesheet. I think there are a few others around here that still do this.

[1] https://armincat.com/

Comment by bookofjoe 1 day ago

[2] https://bookofjoe2.blogspot.com/

Comment by lifeisloving 1 day ago

This one is particularly bad, like they probably made $7500 (not much tbh) for p1, atleast spend a few hours to write the blog post.

Comment by zx8080 1 day ago

For who?

Comment by loveparade 1 day ago

For the LLMs so they get new training data

Comment by metalspot 20 hours ago

exactly. nobody is reading it so why waste time writing it.

Comment by Kuyawa 1 day ago

$5000 is a joke. They pay millions to their top devs and still have more holes than a colander so a nice guy shows you a huge opening in your platform and all you can give him is a pat on the shoulder? Did you fire your top devs and hire that guy for a million bucks plus options? Did you sue the security firm you paid millions to audit the issues?

I really hope they get hacked and crash down burning in flames like the Hindenburg

Comment by cbsks 1 day ago

I wonder if the new economics of finding bugs means that they pay the amount they think the tokens cost to find it.

Comment by aprilthird2021 1 day ago

Makes negative sense. If it were that easy, OpenAI wouldn't have any bugs bc they could just run agents to exploit their own stuff and then fix all their bugs.

This bug could not be found by just asking an AI "find bugs in ChatGPT", that's why it was reported through a bug bounty programs and that's why it should be worth more than $5k

Comment by stavros 1 day ago

Over the probability that you'll find something.

Comment by tsunamifury 1 day ago

I think you know that the mentality of a company like this is “security is a risk but not a reward and whoever captures the reward kills everyone else”

Comment by hdjqnd8382bd 1 day ago

[flagged]

Comment by mannanj 1 day ago

actually, your comment is naive. clearly you don't understand game theory and economic incentives. if a company doesn't reward good behavior, you'll see bad behavior.

but then again since you're not naive, you already know these AI companies like openai already act evil and like bad entities.

I conclude therefore, per your words "you are the last one anyone should consult with, basically about anything"

Comment by breezybottom 1 day ago

Which game theory model specifically applies here?

Comment by usernametaken29 1 day ago

Now what about those millions of lines of code? Wasn’t the agent swarm supposed to take care of it? All jokes aside this is exactly what happens as you build larger monoliths, system complexity and bug surface increases logarithmically… so in all seriousness you want to reduce LOC as much as possible. Maybe this can teach OpenAI a lesson to produce software that’s correct and adhere to established engineering standards over sensationalised “we built so much code” promotional material.

Comment by torben-friis 1 day ago

Did you mean exponentially? Logarithmic increase doesn't sound too bad.

Comment by dahart 1 day ago

I’m sure exponential is what parent meant, but geometric (edit: polynomial) is what actually happens, not exponential. Code complexity increases linearly when it doesn’t interact with other features, and geometrically when it does. In the worst case, complexity increases as the square of lines of code. That is still very, very bad, but it’s not exponential.

Comment by ebertucc 1 day ago

What's the difference? From what I'm reading online geometric is just the discrete version of exponential

Comment by dahart 1 day ago

No difference, you’re right. I meant polynomial not geometric. :P

Comment by cowlevel 1 day ago

geometric/polynomial: x^2, x^3, etc

exponential: 2^x, 3^x, etc

Everything in the second group eventually grows faster than everything in the first group, no matter how small and big you make the fixed numbers, so they are two different classes.

Comment by dahart 1 day ago

Geometric is exponential; I goofed above. Geometric and exponential are sometimes used interchangeably, but sometimes people use geometric to refer to discrete intervals and exponential to refer to continuous functions.

Comment by codechicago277 1 day ago

“Square of the lines of code”

Sounds exponential

Comment by dahart 1 day ago

That’s quadratic, not exponential. 2^n != n^2

Comment by cowlevel 1 day ago

2 is an exponent but exponential growth means the exponent grows. 2 does not grow, it's always between 1 and 3 (unless you use an AI to do maths).

Comment by teiferer 1 day ago

And that's why sw eng interviews often have some basic math sprinkled in.

Comment by 1 day ago

Comment by Pannoniae 1 day ago

it's not like microservices fare better, when you introduce them, you've traded a hard problem for a hard problem and distributed computing...

Comment by wg0 1 day ago

Our industry has wasted hundreds of billions of dollars on this shit show called Microservices.

99.9% of shops need a monolith with same monolith having entry points for background workers and crons.

Comment by rglover 1 day ago

I like LOEC: Lines of Effective Code.

Much easier to reason and make decisions if the bar is "yeah it exists, but is it effective relative to its intended purpose?" This weeds out buggy code as well as under performant code. If it's not effective (broken, too slow, etc), it either needs a refactor or thrown on the scrap pile.

Comment by elendilm 1 day ago

Reducing attack surface is a lesson for everyone. Newcomers are impressed by massive piles of code.

Seasoned developers are unimpressed, or even disgusted, by the same massive piles of code.

Comment by someothherguyy 1 day ago

i don't think LoC is the metric. sometimes fewer lines of code results in a larger attack surface through type coercion / widening, poor abstractions, hiding or stuffing behavior (modules that do everything), etc. to me, making code easy to understand, test, fuzz, and verify is more important. sometimes that requires more lines of code.

Comment by cowlevel 1 day ago

More abstract code probably results in more high-abstraction-level bugs per LoC, but also less low-abstraction-level bugs so I'd guess it about cancels out to an even rate per LoC.

A single line in C++, Rust or Haskell can have a really subtle logic error. A single line in assembly can't, but it has that same likelihood of having a register allocation error instead, and when you add up all the many lines of assembly you probably still wrote the same number of logic errors as the C++ version.

Comment by brazukadev 1 day ago

LoC *IS* the metric. It is obviously not perfect tho but still the best one to compare software surface. Including to compare the same solution in different languages and platforms. The less code you need to manage, the less code you need to change and the less code you need to hold in your head to understand it (or in the LLM context nowadays).

Comment by pessimizer 1 day ago

> Now what about those millions of lines of code? Wasn’t the agent swarm supposed to take care of it?

What's the joke? They're literally saying, every day, that this is true to a civilization endangering extent. They're making demands based on this, and politicians are taking up those demands.

And now they're paying bullshit $5K bug bounties for account-stealing exploits on their own platform. They should have paid more just the make the bug seem more grandiose; I don't know who let them get away with this - this is what happens when CEOs run their own PR I guess. It's basically an public announcement that they're total fakes.

Comment by 6thbit 1 day ago

Appreciate the incremental write up building up the scope of the poc.

I like the chain of regular old school things used to achieve it like post message, JSONP, iframe, Img tags.

Comment by hridyaagrawal 1 day ago

$5000 is a joke for a "P1" vuln

Comment by itsafarqueue 1 day ago

My theory is vulns will never pay out big especially now, no matter how serious. Imagine they paid out $50k+ — it’d spawn an industry doing nothing but.

Comment by verteu 1 day ago

For a company that truly cares about secure software, that seems like an ideal outcome.

Comment by htk 1 day ago

5k makes me think the next guy is gonna sell his exploit in the black market instead.

Comment by incognition 15 hours ago

[flagged]

Comment by 15o9o 1 day ago

we are getting to the point of “why have passwords?” Right? At some point soon we have to move to the honor system of just please don’t login as me?

Comment by qup 1 day ago

My latest project emails a magic link.

I'm trying not to have any important data to even lose, when I eventually get hacked.

We can all turn up our noses, like the other commenters in this thread, but every one of them is exposed. Negligence is a matter of degrees.

Comment by latexr 1 day ago

> I'm trying not to have any important data to even lose

This is the way. Respect user privacy and keep as little data as possible, and not only do you become a less attractive target, if you are targeted the outcome won’t be so bad.

Comment by dataviz1000 1 day ago

> please don’t login as me

Please don't drain my $6.5 billion in Dogecoin.

Comment by chamomeal 1 day ago

Been sayin this for years. Everybody in the world world: just BE COOL. Solves like every internet problem at once.

Comment by 15o9o 1 day ago

we all joke but spiritual communities have been saying this is the way for years. Like the law of one stuff.

Comment by atrokhymchuk 1 day ago

if only they had some sort of cybersecurity state of the art thing...

Comment by jwpapi 1 day ago

I think there was another HN discussion about it re bounty value and as far as I remember the issue is not that big companies are greedy in that case, but rather that over time it’s strategically better to have lower bounties so less people are interested in it.

5 or 50k doesnt matter to them. And this bug in terms of severity is very high imho.

Comment by aprilthird2021 1 day ago

But couldn't they just turn around and sell this hackers for much more money?

Comment by monster_truck 1 day ago

only $5000 for this, lmao. It should be $50k out of pure shame. If they don't make these things hurt just a little they're never going to take following best practices seriously enough. There will be 9 more anyways.

Meanwhile the actual user of the account probably has to sign in using a yubikey because they get "CyBeRsEcRuItY aLeRtS" when they try to update node in their project to fix a crash and have to switch to daybreak blue to continue (or attempt to trick the normal models into doing it and risk getting autobanned)

Comment by joshuamoyers 1 day ago

unrelated to this, but this company randomly reached out disclosing a vulnerability to us (test environment, synthetic tenant). on the one hand, thank you, it’s patched. on the other - we do not have a bug bounty program and this is not welcome. having agent swarms randomly breaking in to environments and then marketing their services in the disclosure is not an ideal sales tactic, in addition to being illegal.

Comment by uniq7 1 day ago

I don't want to defend them, but the alternative is that they do nothing and you don't know about the vulnerability until a bad actor exploits it (something easy and cheap nowadays).

I'm confused, would you welcome that more?

Comment by pamcake 1 day ago

False dichotomy. There are plenty of other possible approaches outside of "having agent swarms randomly breaking in to environments and then marketing their services in the disclosure" and "do nothing".

Comment by uniq7 1 day ago

Tell me one.

Comment by metalspot 20 hours ago

> this is not welcome

tough luck. with an attitude like that yngmi.

> illegal

do you think the russians, chinese, israelis, iranians, north koreans, and a million other profit motivated adversaries gaf about what is legal?

Comment by jareklupinski 1 day ago

> able to take over anyone's account simply by sending them a link

dont click links people

that'll be $500k for the security training pls

Comment by kro 1 day ago

If they or their company use email "security" software that "scans" all links it's not necessarily the users fault / they don't in fact click. The mail is auto-confirmed the second it hits the inbox.

We've had to change our registration confirmation mails too recently to not confirm on GET only, but with another form - in hindsight it was stupid before but many services do that.

Comment by dpark 1 day ago

Security software fetching links will not trigger this behavior. It needs to be done in the context of the actual ChatGPT user so that a JavaScript payload can run.

If merely GET on the link was sufficient to trigger the behaviors, the adversary generating the link would just GET it themselves.

Comment by kro 1 day ago

Yes, if it requires the user session (xss related), like the article, there is no problem.

I was replying more generally about the "don't click links" advice and cases of server generated confirmation-token links (not saying this was the case at OpenAI)

Comment by weird-eye-issue 1 day ago

> We've had to change our registration confirmation mails too recently to not confirm on GET only, but with another form - in hindsight it was stupid before but many services do that.

This has been a thing since forever lol

Comment by gokulravi992 1 day ago

[flagged]

Comment by dist-epoch 1 day ago

I wanted to check your website from your profile, but then I remembered your advice. Thanks!, dodged a potential bullet

Comment by bossyTeacher 1 day ago

Also many emails from tech companies today:

- "sign in via one-time code."

- "Click here to see our latest update".

Comment by jareklupinski 1 day ago

dont click links

250k refresher fee

Comment by pkthunder 1 day ago

Click this link to be told not to click links.

Comment by atrokhymchuk 1 day ago

hey, don't dump the market

Comment by kotaKat 1 day ago

"Scan this QR code!"

Comment by 1 day ago

Comment by TobTobXX 1 day ago

... says IT and then it sets Outlook to mangle link to abominations like [0]. "Check if the preview shows something different or suspicious."

[0]: [redacted]

Comment by Antipode 1 day ago

I've worked at a company that did link mangling on everything except simulated phishing emails. The effect was that people were trained to trust the mangled links.

Comment by 1 day ago

Comment by extraextra 1 day ago

God I hate those

Comment by Razengan 1 day ago

Thanks! Here, click on this link to get paid

Comment by gib444 1 day ago

[dead]

Comment by pllbnk 1 day ago

It's exhilarating (for the lack of the knowledge of a better word) to see OpenAI and Anthropic starting getting hacked and getting a taste of their own medicine. Brings back hope that actual software engineering is not dead yet.

Comment by jiveturkey 1 day ago

Hex aka Parameter is an irresponsible company, on my blacklist. after they did an unsolicited pentest on us, we scheduled a call. joke of a team.

hilarious that their headline message, on every web page, is `Backed by YC`.

also, the product is way behind their competitors anyway.

really cool exploit they found here, but that doesn't make them a company worth doing business with.

Comment by Zeruxe 1 day ago

5k is nothing for the amount of headache you just saved them..

Comment by j9feng 1 day ago

Claude or Gemini didn’t find the vulnerability on ChatGPT?

Comment by Yieldz 1 day ago

Sigh. With how good Openai are good in programming frontier agents, they still have weak security. Hmmn, i'm short of words.

Comment by TacticalCoder 19 hours ago

I thought we had AIs and coding was solved?

How comes the very companies creating, supposedly, the best LLMs in the world have such blatant one-click security holes?

Something doesn't add up: sounds to me like coding ain't entirely solved yet after all...

Comment by krapp 19 hours ago

>sounds to me like coding ain't entirely solved yet after all...

Someone's getting a lump of coal from the Basilisk this year...

Comment by nickphx 1 day ago

why didnt the black box of bullshit company detect this? for a group that touts every inane 'discovery' with hyperbolic bullshit, they really fall short on delivering anything of actual value, just rehashed bullshit.

Comment by jiang-li 1 day ago

[flagged]

Comment by msarda_23 1 day ago

[flagged]

Comment by gewetensleegte 22 hours ago

[dead]

Comment by marincoco1 1 day ago

[flagged]

Comment by frank00001 1 day ago

[flagged]

Comment by marwaneet 1 day ago

Another classic OAuth implementation failure. Mismanaging the state parameter or leaking tokens via referer headers in redirects is Security 101. Wild to see this in a production environment of this scale.

Comment by brabel 1 day ago

The post seems to be talking about a html injection via a postMessage call where the receiver didn’t check who sent the message, which is a well known vulnerability entry point. Did I miss where OAuth had anything to do with this??

Comment by s5300 17 hours ago

[dead]

Comment by steve1977 1 day ago

Production scale is not production grade I guess...