Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

Posted by speckx 3 days ago

Comments

Comment by Cyan488 3 days ago

> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice.

I'm not sure "worked properly" and "as intended" accurately describe this situation.

Comment by vb-8448 3 days ago

In italian we say "l'operazione è riuscita perfettamente, ma il paziente è morto" -> "the surgery was a complete success, but the patient died"

Comment by embedding-shape 3 days ago

Both this and what Meta said reminds me of "Clarke and Dawe - The Front Fell Off" (https://www.youtube.com/watch?v=3m5qxZm_JqM)

I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.

Comment by vb-8448 3 days ago

> It's like there is no humans working and writing there anymore.

Don't know if AI is to blame, but I've used to see these kinds of nonsense post-mortems even in the pre-llm era, and it's always due to some internal fighting ongoing between various departments.

Comment by daveshistory 3 days ago

Where do you think the LLMs learned it from...

Comment by saturnite 3 days ago

"Who taught you how to do this stuff?"

"You, alright! I learned it by watching you!"

Comment by daveshistory 2 days ago

Pretty much. The most depressing thing about the bland slop produced by LLMs is realizing that this is what they "think" of us.

Comment by rothfuss 3 days ago

I was reminded of the Murray Walker quote. “There's nothing wrong with the car except it's on fire”

Comment by lelandfe 3 days ago

My dad says, "But other than that, Mrs. Lincoln, how was the play?"

(Usually said jocularly when everyone is at their most upset, e.g. a vacation ruined)

Comment by RRWagner 3 days ago

A friend said at one of those moments, "And other than that, how was the play Mrs Lincoln?" And the 3rd person replied, "I don't know, I've never seen the play 'Mrs Lincoln'"

Comment by thrownthatway 3 days ago

[dead]

Comment by maxall4 3 days ago

“The strait of Hormuz is open so long as Iran does not fire missiles at ships.”

Comment by dylan604 3 days ago

"The numbers will go down as soon as you quit testing"

Comment by trumpdong 2 days ago

The actual quote was "if you don't test, you won't have cases"

Comment by Arainach 3 days ago

> like there is no humans working and writing there anymore

Meta has never been a place for people with empathy to thrive or succeed. They literally enabled a genocide. Despite being warned by internal employees, profits were more important.

Comment by emayljames 2 days ago

Which one. They have several under their blood soaked belt now

Comment by 3 days ago

Comment by RobotToaster 2 days ago

Very rigorous software engineering standards.

Comment by prinny_ 3 days ago

Does it matter if the response is tone deaf or simply misguided? I am a bit nihilistic here, but in one week absolutely nobody will be talking about this. Are the affected individuals going to abandon instagram? Are people going to reduce their usage out of concern for the safety of their accounts? Nothing will happen, hence there is no need for actual humans writing a good, well intended response.

Comment by justinclift 3 days ago

> in one week absolutely nobody will be talking about this.

In news media, sure. But in IT teams around the world people will be referring to this (the exploit opening stupidity) for years as how NOT to do things. :)

Comment by skyfaller 3 days ago

Everyone has a limit to how much bullshit they'll put up with. This could be the last straw for some people to finally quit Instagram. I quit Instagram, Facebook and all other Meta properties in 2025, after complaining about various problems for years. Other people may quit temporarily and then return, but any time they spend away from Instagram may give them experience that will help them quit permanently later.

Comment by vb-8448 3 days ago

> Does it matter if the response is tone deaf or simply misguided?

I agree with you that in a week nobody will be talking any more, but I'm pretty sure it's a GDPR data breach, and they can have some trouble within EU.

Yeah, they probably don't give a fu.. about EU, but if the response doesn't matter at all why did they spend time on it?

Comment by redbell 3 days ago

Haha.. This reminds me of a classic Windows MessageBox meme that goes: "Operation failed successfully!"

Comment by panzi 2 days ago

We have the same saying in German: Operation erfolgreich, Patient tot.

Comment by unmole 3 days ago

"Operation succesful, patient dead" is a common saying in India.

Comment by eesmith 2 days ago

I found an English use from 1883 - https://archive.org/details/argonaut131883sanf/page/n391/mod... .

> The creosote in toothache drops administered to a New York boy cured the pain, but killed the boy. This recalls the entry in the register at Bellevue Hospital, which reads; "Operation successful. Patient died."

The Argonaut, San Francisco, December 22, 1883.

Comment by 3 days ago

Comment by methyl 3 days ago

Maybe it was euthanasia?

Comment by raffael_de 3 days ago

"operation successful, patient dead."

Comment by thrownthatway 3 days ago

[dead]

Comment by nkrisc 3 days ago

The tool worked correctly and as intended, but due to a bug it did not work correctly nor as intended.

Comment by thih9 3 days ago

To be fair, that quote in the original article could have more context. By "The tool" they meant "AI-assisted support tool"[1]; perhaps they meant that the issue was not an AI hallucination inherent of the tool, but a fixable bug.

[1]: https://www.documentcloud.org/documents/28202858-meta-ai-ag-...

Comment by nvme0n1p1 3 days ago

In that case, the statement is so meaningless as to be useless. Why should we care how Meta splits up their microservices? The tool still failed. They just want to redefine the "tool" as something else, anything else, to avoid having to admit something negative about their precious AI.

> The LLM correctly generated tokens according to user input, however due to a bug in a separate code path, the system did not properly verify the email address

> Nginx correctly handled the user requests according to the HTTP standard, however due to a bug in a separate code path, the system did not properly verify the email address

Comment by csallen 3 days ago

I mean, I think many of us are curious and enjoy hearing more details about how and where bugs like this occur. What's wrong with that?

Comment by nvme0n1p1 3 days ago

I'd love to read a proper technical post-mortem, but this obviously isn't it. It's a carefully-worded statement from a lawyer meant to minimize liability and reputational damage to the company.

Comment by albedoa 3 days ago

There is nothing wrong with that, and nobody is saying there is. In fact, it is exactly what is being requested here!

Comment by zahlman 3 days ago

It seems to me like they're saying the agent made the tool call they expected, but the harness didn't reject it like they expected it to.

Comment by eterm 2 days ago

But it sounds like it's not even a harness issue if they have a process where they send a reset email to an address that isn't associated with the account.

This isn't (just) a validation issue, and shouldn't be at the harness level.

Comment by theptip 3 days ago

Sounds like they are saying the agent did not malfunction, and this vuln could have been triggered by a human support agent too.

Comment by mikeocool 3 days ago

Kind of interesting that LLMs are basically being sold as having “human-like” reasoning capabilities, but in this case when “obamawhitehouse” asked to have it’s password reset sent to bob12345667@gmail.com the LLM didn’t question it and just triggered the process that happened to have a bug.

Humans support agents certainly fall prey to social engineering all the time, but I can’t think of a case where it was done on this scale so easily.

Comment by trehalose 3 days ago

It probably could have been, but how likely is that compared to with the AI agent? I'd assume (and I'm ready to look like an idiot if I'm wrong) that the humans are trained to send the verification code to the email address on file, rather than any address the client asks them to. I'd certainly assume most of them are more afraid of the consequences than the AI is.

Comment by theptip 3 days ago

For sure. Social engineering attacks on human support staff are common and well known, but the skill floor is non-trivial; you need to actually be able to convince a human of your ruse.

Having a support agent likely made it easier to enumerate the vuln, and certainly made it easier to scale out exploitation once it was discovered.

Comment by dd8601fn 3 days ago

I think they’re blaming a tool function so as not to admit the overall agent process was shit.

But it’s irrelevant, outside of PR. We know at least THREE bad components to this process and they were constituent parts.

Comment by TZubiri 3 days ago

I get the joke, but it's a relevant nuance that the new code, the chatbot, did not have 'the bug'. I still think that the mistake and head that should roll should be the one that published the chatbot.

But it's important to acknowledge that there was a 'bug' in an underlying tool and not in the chatbot, and still PIP/fire those responsible for publishing the chatbot and exposed an otherwise internal tool to the public, and not those that introduced the 'bug' to an internal tool.

Comment by srdjanr 2 days ago

Why should the chatbot team necessarily take the blame? For all we know, they could have got approval from the tool team to make it public, and passed additional security review for making it public.

Also, why fire anyone after a single mistake?

Comment by TZubiri 2 days ago

I did mention PIP/fire, but to be fair, this looks like the worst security issue in the history of Meta, a company known for an almost impeccable cybersecurity clean sheet.

So yeah, firing somebody or a group of people is on the table. Especially when like 10% of the company was fired last week for unrelated reasons. If you are gonna do it, fire the people who slash the value of your company by billions of dollars.

Comment by cyc116 2 days ago

How not to do blameless postmortem lmao. Non of the the engineers involved in this incident had anything to do with the company-wide layoff. I'm deeply sorry if the layoff affected you. But blame firing/piping more engineers for an incident should NOT be on the table. The negative sentiment towards meta engineers on this post is just wild.

Comment by TZubiri 2 days ago

>But blame firing/piping more engineers for an incident should NOT be on the table.

There has to be a level of fuck up where a resignation is appropriate, maybe this doesn't meet your bar, but surely you recognize that there exists a limit of incompetence that proves that one is not up to the demands for the job.

I used to be on your camp, blameless postmortems, the truth is more important than assigning blame and in all likelihood it's a systemic problem. But with time I realized two things, 1 there's actually incompetent people, 2 if you wrongly get blamed and you don't blame someone else, then it's your head that rolls, hate the game not the player, you have to assign blame to someone else if you are accused.

Comment by nico 3 days ago

That sounds a lot like the justifications Claude and ChatGPT give when confronted about something they did wrong, or when asked to provide a customer support response about software issues

Comment by dmoose 3 days ago

I've lost track of the number of times Claude has basically said "it was like that when i got here" in the face of a clearly bogus choice and easily disproved explanation.

Comment by AlienRobot 3 days ago

They should add a feature called "auto-really" that just automatically says "really?" after the chatbot answers a question to check if it's going to 180 upon this tiniest bit of scrutinity.

Comment by lou1306 3 days ago

You joke but this is almost literally what Chain-of-Thought does, at least in the early days. They basically just added "Wait," to the model's output and fed it back to the model iirc

Comment by AlienRobot 3 days ago

This can't be a trillion dollar industry...

Comment by trumpdong 2 days ago

It's the ELIZA effect.

Comment by smcin 3 days ago

"Really?"

Comment by nosioptar 2 days ago

I tried asking about how my state treats violations of updoc. It replied with a long ass wall of text about how serious the Updoc statutes were in my state and how judges punish harshly.

I pointed out that updoc is nonsense and asked why it didn't catch that. The answer was that it was my fault for giving it bad info.

Comment by armoredkitten 2 days ago

what's updoc?

Comment by nosioptar 2 days ago

Not much, how're things going for you?

Comment by tommit 1 day ago

got em

Comment by nsvd2 1 day ago

There is no difference, from the model's point of view, between code it wrote and code someone else wrote. It's all just context.

Comment by Chu4eeno 3 days ago

You need to hit the retry/regenerate button more, it's there for a reason.

While the "stochastic parrots" thing is a bit overblown, IME most LLMs tend to surprisingly different responses even without changing the context, especially if they're hallucinating or doing something "wrong".

Comment by Cpoll 3 days ago

The argument here is that the AI is a glorified input page. The input field asks for your username and email and sends it to a backend function. Such an input page is working as intended.

The problem is when the backend function doesn't verify that the email matches the username.

Comment by dgoldstein0 3 days ago

Why on earth would the backend function even take an email?

Or perhaps said different: use the submitted info to identify the account; send any sensitive messages (recovery codes, password resets whatever) to only the contact info on file. If the chat bot can send such email it should do so via an API that sends only to contact info on file for the associated account and not to an email that's provided by the bot.

Comment by duskwuff 3 days ago

> Why on earth would the backend function even take an email?

In principle, it could be designed to do so to handle cases where a new email address has been confirmed out of band, e.g. for an account representing a company or a political office. But that's a relatively unusual situation, not something you'd want to be available to every user writing in. (Even if you had an all-human support department, this sort of functionality would only be available to a select few agents.)

Comment by Cpoll 3 days ago

Some sites do this to prevent password recovery spam; you need to provide two pieces of information. Ideally not telling the client if they wrote the wrong email, that'd be a security issue of its own.

Comment by Polizeiposaune 3 days ago

When such systems are hooked up to a web page they often will ask which contact should receive the reset code

(Pick one:

"send text to number ending in -1234"

"send text to number ending in -5678"

"send email to jo......th@gmail.com" )

Comment by jgalt212 3 days ago

Fair enough. Never trust client-submitted browser form, but always trust LLM-submitted form.

Comment by lou1306 3 days ago

If the backend function was so poorly coded to allow such a gargantuan security hole, then it is an even worse problem. Basically Meta is throwing its own engineers under the bus so that its AI chatbot can save face. Scary stuff.

Unless the backend was _also_ vibe-coded, in which case it is still an AI problem.

Comment by oenton 3 days ago

Okay, I hear you. I do. From a technical viewpoint, that may very well be how their systems are implemented. But this still doesn't answer the question of why the fuck this matters to these states' AGs and the people they represent.

Comment by 3 days ago

Comment by ChuckMcM 3 days ago

Read that as "worked as written" and "we disclaim any consequential or incidental damages and do not warrant this software."

I continue to believe we could fix a lot of things in the US if we updated the UCC[1] to disallow 'disclaiming liability on software used in a product.'

[1] Universal Commercial Code -- https://www.law.cornell.edu/ucc

Comment by jjmarr 3 days ago

I've always wanted to expose myself to unlimited legal liability by distributing open source software.

Comment by Terr_ 3 days ago

That seems like a false-dichotomy between two extremes when there's all sorts of space in the middle... It's also assuming developer-to-developer tools would have the same rules and exposure as in service-to-consumer.

If I sell a physical motor (let alone plans for one) I'll have some liability for things like it Not Exploding. If someone buys a dozen of those motors to assemble a tragically unsafe "rollercoaster" of their own design and construction, I'm almost certainly not responsible for any terrifying decapitations.

In other words, most of the world already does not rely on the issuance of "Get Out Of Infinite Liability Free" cards.

Comment by ChuckMcM 3 days ago

Exactly this. (and it is a false dichotomy to argue infinite liability).

To Terr_'s point, if you were publishing open source you would also publish exactly the things you intended it to be used for and anything else would violate your warranty (possibly implied) that it does what the documentation says it does.

There is a huge amount of tort law that covers exactly when it becomes a problem for you the creator vs you the user in your own project. And that liability is also based on once you know something bad could happen you make an effort to notify people[1].

[1] https://www.cpsc.gov/Newsroom/News-Releases/2026/Clorox-Agre...

Comment by Ajedi32 3 days ago

Software can be copied infinitely, so even $1 of liability is effectively infinite since an unlimited number of people can potentially use it and sue you when it blows up.

Nobody's going to be distributing software on the internet for free if the cost of insurance alone precludes that.

Comment by ChuckMcM 3 days ago

This is not how liability works, anywhere. So I write a piece of code that "makes your screen do cool things" and it causes the power supply to fail on those screens. Someone reports that bug to me and I check it out and say "Oh, shit it does break power supplies." Then I immediately put a notice on and in the code that says "WARNING: This code will break the power supply of your montitor." And I put that warning in the repo. And if there is a Discord or a mailing list I tell everyone "Hey, this is important, if you run this code it can break your monitor."

Guess what, I'm not liable for the damage. Why? Because I immediately responded once I knew that it could, I made a good effort to warn people who might already have the code of the risk, and I made it clear in the code that this risk is there.

Ever wonder why you get a booklet of warnings when you buy a product with even really stupid things like "Don't clean with gasoline" warnings? That's because once you have discharged your duty to warn you are not longer liable in what happens if someone ignores your warning.

The flip side is also true, you cannot say in your product both "Hey this product does these cool things" and "We don't warrant the product to actually do anything." This is especially true if there is money involved (like your user paid your some $ for the product.) There is always an implied warranty that the thing will do what you says it will do, which exists as long as the user has heeded all your warnings.

Comment by fc417fc802 3 days ago

I broadly agree with you but TBF to the earlier comment consider what would happen if a FOSS author did something wrong and was found to be liable. How about curl for example? That sees use in car infotainment systems among other things and cars can be pretty expensive and there sure are an awful lot of them. The point is that we should be able to accommodate someone pushing a hobby project to github under a permissive license while also imposing liability against developers in instances where money changes hands or where someone's work involves interacting with the physical world.

Comment by trumpdong 2 days ago

The EU CRA handles this by putting liability on someone who integrates FOSS into a product instead of someone who wrote it. Because it doesn't make sense to put liability for unforeseen downstream uses on someone who gave away something they made in their spare time. Now, if it was a virus, you're still liable for distributing a virus.

Comment by Ajedi32 1 day ago

Yes, when you're selling a product you can price the risk of lawsuits into whatever you're charging customers. You can't do that with free software without making it no longer free.

"No problem: just don't get sued" only works if legal battles are free and/or the law makes it so blatently obvious that you're not liable that nobody would bother to try.

Comment by ChuckMcM 3 days ago

I realize this is drifting off topic, and happy to talk more in email (address in profile), in the interest of sharing a bit more, consider this statement you paraphrase:

"a FOSS author did something wrong and was found to be liable"

In fairness, I not sure the earlier commentator really understood what they were saying, at least not as far as legal liability is concerned.

The FOSS author simply wrote some code and shared it right? That is their 'action' can you think of ways that does direct harm, which is to say they published their code, and with nothing else happening someone got harmed? One way that can cause harm is the FOSS author publishes a trade secret[1] or access credentials of a third party. In both cases they could (and would) be sued by that third party. But absent that, I'm having a hard time coming up where simply the existence of most code causes someone else harm.

So to get to harm we have to add another person, that person somehow applies the code, and in that application harms another person. Our FOSS author might be sued as being contributory because the person who caused harm might not have done so if they didn't have access to the code. To prove that, the plaintiff would have to prove that the FOSS author knew that the code could cause harm if used in this way, and encouraged or otherwise abetted the person who did harm to use it in doing the harm. That can be a hard standard to reach[2].

In your car example, it would be challenging to prove that Daniel Stenberg wrote curl so that you could use it to brick car infotainment systems. But it would be easier to prove that a manufacturer that incorporated FOSS code and didn't check their system for risks like this should be found liable.

Liability accrues first to the party that did the action. Secondary liability can reach out to suppliers[3] of things used in that action. This is also civil law rather than criminal law and so it works a bit differently in terms of evidence standards and penalties.

[1] We can make a joke here about badly formatted code, but hopefully we're in a agreement so far. A real example was the DVD decoding software that included the key for decoding encrypted DVDs.

[2] Not that people might not try, its too easy to sue. There have been cases where someone wrote some code that was later used in a weapon (and example might be Ardupilot software in drones used to kill Russians). But even in that case, the courts in the US at least have consistently found that if it is not the primary purpose of the software to do harm, then the author is not liable.

[3] Unless you're a gun company as Gun companies have managed to keep themselves from being found liable for people using their guns to do harm. But there is also lots of interesting case law there too which might help inform.

Comment by fc417fc802 3 days ago

That's a really good point. Where I remain at least somewhat concerned is for example suppose that one day curl pushes a terrible bug to production that results in all sorts of nasal demons flying out of client devices. Is this free code that was picked up off the side of the road thus zero liability? Or is this a trusted product written and maintained by a professional that has stood the test of time thus there might be liability because there's an assumption that official updates will be fit for purpose?

Now if I were running a small business I might choose not worry about the tail risk of my product causing a few million dollars in harm or (more likely) I'd have insurance to cover that. But someone tossing code along the side of the road presumably doesn't have (and doesn't want to think about) insurance and meanwhile the tail risk has become nearly unbounded thanks to the effectively arbitrary number of deployed instances.

I think there's also some benefit to having a big fat NO WARRANTY clause at the top of the license file because it might give you a better chance of a summary dismissal (or even deter the other party from trying in the first place) since as we all know the process itself can be ruinous even if you eventually prevail.

Which is all to say that I share your view. Willingly negligent vendors that cut costs by omitting security while viewing the resultant mishaps as an inescapable reality ought to be held accountable. But I think it would also be a good idea to add an official exemption for software that's made available free of charge. It seems like if you pick something up off the side of the road any mishaps that follow from that should necessarily fall to you.

Comment by aleqs 3 days ago

There's a pattern I noticed, especially on this site, where people claim various VC/ad/tech dark patterns, enshitification, privacy violations, dishonest marketing, etc MUST be allowed, otherwise open source or 'the internet' will face some sort of existential risk.

No bro - open source and the internet existed long before SV tech parasitism did and will exist long after.

Comment by ChuckMcM 3 days ago

I don't disagree, that pattern exists, but it is essentially true. Just not in the way the folks saying it is true understand it. If the "VC/ad/tech dark patterns, enshitification, privacy violations, dishonest marketing, Etc." wasn't allowed then their job might not exist. That can be true. What is missed is that if there is value in the thing, then it will exist.

When I reflect back to someone making this argument by saying, "So your argument is that you make your living as a pick pocket, but if pick pocketing is made to be illegal, you won't be able to make a living." Which of course would only be true if they only thing they could do was 'be a pick pocket'. Its a very common rhetorical technique to argue that the status quo cannot be changed. All the arguments that "you'll put all coal miners out of business if you require only green energy" And yet the people, the miners themselves, will likely be fine. The firms might not, but there are other firms that could exist.

This isn't a new problem, or one specific to this web site, although it does get disproportionately hit because so many technology companies saw what Google started in the 2000's and said, "Man there is soooo many ways to get money for this." rather than, "Is this a reasonable way to make money? Sure it is 'perfectly legal' but is it right? Is it moral?" The type of person who thinks that something is "Only illegal if you get caught" is neither moral nor particularly concerned about what is right. And we got a lot of that type.

Comment by oenton 3 days ago

"Its a very common rhetorical technique to argue that the status quo cannot be changed."

Thank you for putting this so eloquently into words. This rigid thinking is also common in topics such as working conditions, collective bargaining, on-call time, parental leave, healthcare, and effectively (unintentionally or not) shuts down conversation.

I've come to realize the objections from people who think this way all effectively boil down to 'Be grateful for what you have because any alternative would be worse.' But if you pry and ask that they expand you'll find there really isn't any there there, because it's black and white thinking. It isn't rooted in fact, it comes from fear. I sure hope we haven't collectively forgot how to even imagine a system that functions better than the one we have today.

Comment by ChuckMcM 2 days ago

Thanks. For me, I was in debate club in High School and that included basic rhetoric. In college I took an argumentation class as a non-engineering elective. The most useful thing this class taught (for me) is how to 'see' the argument, and as a consequence see how it is constructed. Throughout my career it has been especially useful in "political" situations at work. Not everyone argues in good faith, and being able to spot those who are not is valuable.

Comment by Terr_ 3 days ago

With respect to the need/impossibility of change, the "Politician's Fallacy" seems related:

1. Something must be done.

2. This is something.

3. Therefore this must be done!

Comment by aleqs 3 days ago

Very well put.

Comment by skywhopper 3 days ago

[flagged]

Comment by jjmarr 3 days ago

The United States/Canada don't have a "loser pays" rule, so this exposes me to legal fees.

Right now, any lawsuit against me can be dismissed on summary judgement because even if my software causes harm, that's not a legal wrong to the extent I've disclaimed liability.

If you adopt any fact-specific standard for liability, that needs to be adjudicated in a trial. The legal fees alone would surpass the actual liability.

That creates huge leverage for the party with more resources. That kills hobbyist open-source development, since if your project takes off but a large enterprise finds it defective, they can threaten to sue you to enforce the "warranty" you were required to give.

Comment by Terr_ 3 days ago

> That kills hobbyist open-source development, since if your project takes off but a large enterprise finds it defective, they can threaten to sue you to enforce the "warranty" you were required to give.

I think you're assuming some kind of worst-possible outcome that hasn't been proposed and is unlikely to be enacted. To quote from earlier in the thread: "Disallow disclaiming liability on software used in a product."

I don't think that changes your hobby work on a rational-math library or an MVC framework or whatever, since you aren't making a business out of it. It will affect that large enterprise if they roll out their new product "Yearning 4 Mines: Gatcha Gig-work For Kids."

Comment by skywhopper 3 days ago

Ensuring Meta is responsible for its products would not need to assign liability to someone offering open source software.

Comment by trumpdong 2 days ago

They did say a product. Is it a product if you're not selling it or even giving it away but you just made it available for download?

Comment by lukan 2 days ago

Depends on the jurisdiction I think. And if you take donations, the line gets blurry even faster.

Comment by ncallaway 3 days ago

Would that be software used in a product? I don't think that would qualify?

Comment by RobRivera 3 days ago

Oh it was a downstream dependency. The tool worked, it was the downstream dependency. Glory to Arstotszka

Comment by moffkalast 3 days ago

Tool so great, downstream dependency not required! Right?

Comment by ludwik 3 days ago

I like to dunk on Meta as much as the next guy, but I think this makes sense: deterministic verification like this is not, and should never be, the LLM’s job. The tools it has access to should enforce the permissions layer, ensuring that the LLM can never perform actions the user themselves should not be allowed to perform. In this case, the tool failed to do that.

Comment by TZubiri 3 days ago

>deterministic verification like this is not, and should never be, the LLM’s job.

But when humans handled it, this was not as much as a problem. That is, the humans did the job, because they recognized the need to do that job.

Sure sometimes accounts could get recovered if a human was tricked, but evidently it was easier to trick the LLM in masse than humans.

Comment by ajross 3 days ago

> But when humans handled it, this was not as much as a problem.

In fact it's arguably a feature. The ability of support staff to short-circuit nitpicky rules when there's an obvious external validation happening (e.g. you're on the phone with a user who's presenting ID in real time and correlating it with previous use of the account, etc...) makes for better data quality and happier customers.

Obviously, yes, you can then human-engineer an authentication breach. But that was very difficult, because people are "common-sense careful" in a way we haven't been able to tease out of AI yet.

Comment by ludwik 3 days ago

Maybe that’s because I work with agentic AI in my day job, but this seems utterly obvious to me: no reasonable person would ever claim that LLMs are better at keeping secrets or enforcing rules than human employees.

This notice is not about comparing humans and LLMs. It seems that the system was designed in the only reasonable way: with a deterministic permissions layer separate from the agent. But that layer failed to work properly.

So the notice is comparing the difference between how the system was supposed to work and how it actually worked in reality. Normal post-mortem stuff.

Comment by gavmor 3 days ago

The overall system that allowed this implementation is accountable. So why put such a fine point on it so as to exculpate the LLM?

Comment by im3w1l 3 days ago

It helps set expectations for the fix. "The bug was in an external system that has now been fixed" means we it's probably fine going forward. "The LLM got tricked but we are gonna train it super hard not to do that again" means it will break again and again as people find new angles to convince it.

Comment by dbbk 2 days ago

Yes the LLM part is irrelevant here. It'd be just the same if it was a HTML form.

Comment by jffyjcaa 3 days ago

[dead]

Comment by teaearlgraycold 3 days ago

> The tool itself worked properly and functioned as intended

The author of the post is close to the author of the AI code on the org chart

> however due to a bug in a separate code path, the system did not properly verify

The author of the post is far from the author of this "code path" on the org chart

Comment by ofjcihen 3 days ago

Maybe they’re communicating exactly what it sounds like and are just owning up to being complete morons?

Comment by photochemsyn 3 days ago

Our autonomous client-assistance system is managed by a teenager that usually makes good decisions but sometimes makes bad decisions and so all the teenager’s decisions are checked by a minder before being implemented. Unfortunately the minder wasn’t paying attention, so, here we are. However, our teenager is a great kid and did nothing wrong! It’s all the minder’s fault.

P.S. Would you like to have our teenager manage your system too? Terms are reasonable! Of course you accept all liability, so better get a good minder - and no, don’t use an AI as the minder, that just introduces a new failure mode.

Comment by TZubiri 3 days ago

Of course.

What I gather is that this internal tool was used by human support agents, and it was their responsibility to verify the email adresses and general validity of a claim.

But when implementing AGI TM that was overseen, maybe the oversight in the separate code path was a 'bug', but the mistake was making the chatbot obviously, if the separate code path had a bug, then it had become ossified into a feature, and it was internal, not exposed to the public.

This is an external communication, to save face sure, but if this is the internal excuse, it would be absolutely the wrong RCA and it reads as if the one who made the mistake is not admitting they made their mistake. Which to be honest, just making the mistake is enough to get fired, but not admitting it is enough to get ultra fired.

Comment by warmedcookie 3 days ago

Having had my 2FA Facebook account banned 3 years ago because a bot signed up under my email for Instagram (which I did not have), I can confidently say the email verification issue has been a problem for a long time at Meta.

Comment by xyst 3 days ago

It’s a public release prepped/reviewed by the in house legal counsel.

Don’t read too much into it. Facebook wants to face as little accountability and keep the future class action lawsuit to a minimum.

Comment by 3 days ago

Comment by saltyoldman 3 days ago

Isn't that exactly what they said when Cambridge Analytics data gathering happened?

Comment by totetsu 3 days ago

Then ‘ The tool itself’ was not appropriate to the job in the first place

Comment by trumpdong 2 days ago

They're saying: our AI worked perfectly, we just prompted it wrong.

As you do. All AI failures are caused by bad prompting because AIs are perfect.

Comment by Schlagbohrer 2 days ago

No no the tool worked fine, it was the system that failed. They blame society, basically.

Comment by tyleo 3 days ago

Error: Success!

Comment by dylan604 3 days ago

You must work in QA

Comment by laweijfmvo 3 days ago

so how long was the bug there? was there a way to access it before/without the support agent? it feels like Meta will throw anything under the bus to redirect blame from the AI, because that would be the end of their $600B (depending on “which number you want to go with”) experiment

Comment by stephenhuey 3 days ago

What was that mantra? Something about broken software is what they aim for?

Comment by az226 3 days ago

I'm sure. It was not working properly nor as intended.

Comment by dboreham 3 days ago

There should have been a test case for this. There wasn't because most shops don't actually test their product. They do some test theater such as unit testing.

Comment by hsbauauvhabzb 3 days ago

‘Hey Claude, write me a PR statement’

Comment by tomkarho 3 days ago

How very Wernher von Braun of them.

Comment by ncr100 2 days ago

"Marge, there's the truth..." (frowns and shakes head negatively) "...and there's THE TRUTH!" (smiles brightly and nods enthusiastically)

-Lionel Hutz, Simpsons, Season 9 - "Realty Bites"

Comment by cynicalsecurity 3 days ago

This-is-fine.jpg

Comment by 3 days ago

Comment by endofreach 2 days ago

Unfortunately this statement will, in spite of what you identified correctly, likely do its job and divert attention from the fundamental issues we are facing with a technology that has already spread further than anyone can control. From enterprise too lay man. The whole world of computing was not built ever expecting software capabilities like this to ever exist.

I am not saying it's like a nuclear bomb. Rather like the first guns brought into fights the others were perfectly prepared for ti fight with swords and didn't even know yet, about this fascinating invention called a gun. Sounds interesting. Let me inspect it. Oh wow, that's interesting technology. What happens if i push that thing back? Will it re... oops...

Thank god that we have honourable people like altman, zuckerberg, musk. Imagine how bad all this would turn within the next few years, if major decisions were made by self-serving, delusional, greedy egomaniacs...

Of course currently let's first hope those wars and all the tension in societies all over the world, in war or peace, won't explode into something really, really bad. Looking at history, i fear we see how social tension on large scale over time... not saying it's not obvious to almost everyone. So well, let's just keep hoping. Maybe throwing blackbox AI tech into the mix, would surprise and change course of history. Actually, while i am thinking about it, i think i just changed my opinion into the opposite position, lol. Honestly, if it's 50/50 that this will lead to the worst possible outcome intensified, it's still better than just checking boxes following the "humans slowly stumbling into near-extinction experiences 101" handbook. Because just according to that, we're lucky if we're off by 10 years. There must be a big change in humanity and how the world is currently constructed, for all this leading to anything other than what we should expect from history. If we kept all nations busy with huge technological issues, that made all of their personal lifes so complicated, turn every elitists luxury into a burden, busy to defend what they own, while they can't realize, that normal life has changed so much, they now are the ones, frozen in life. They would have no time for conflict.

This sounds totally logical. In any other scenario, it would be pretty insane what we are all doing and entertaining (including me, top10 hypocrite).

I fear it's too late to turn ship, yet we still can jump ship.

---

Especially because now thinking about the thoughts that just went through my head, maybe (technological) disruptions are actually disrupting. But not a status quo of an economic model.

But a pretty clear loop of human nature and "humans in societies". And the more often we disrupt this loop, the more time we get before it's ready to start over again.

And now we have something that has the potential to change all fundamentals so much, that all the major conditions inside this loops iteration become meaningless. The environment changes so much, the state of the checkboxes gets emptied. Cache invalidated. Indices are gone.

Oh, i know how dumb this sounds. I am not even trying to claim anything. I didn't even think about it before, this is just a note of the words that i typed, almost on autopilot. No idea if i believe a part of this could be real. But even thought, just as a mere fictional story, it already entertained me.

Comment by johnyzee 3 days ago

"Meta notified at least 20,225 people that their accounts had been compromised. [...]
The compromises allowed the hackers to take over the person's entire Instagram and any linked accounts, including obtaining contact information, dates of birth, and profile information, as well as the ability to access the person's posts, direct messages, and account activity [...]

the hacks began around April 17 and lasted until this week [...]"

This is staggering.

Comment by Lionga 3 days ago

One can only hope EU gives them a GDPR fine very close to the limit of 4% of global turnover. But when EU is actually need to protect customer I think they will fail.

Comment by mvkel 3 days ago

Incidents like this show how unenforceable GDPR is, and how it's been a net negative for users since its inception. It's idealogical back-patting, toothless when it matters.

Comment by Gigachad 3 days ago

After the GDPR every website added an option to export your personal data and to delete your account. Something most were missing at the time. It was an immediate and massive win.

Comment by hn773746483 3 days ago

Right, but nothing stops companies from refusing SARs on baloney grounds. Complain to a DPA? They tell you to go through ADR or outright ignore you. Complain to Ombudsman? They'll tell you the same. (In my experience, the Dutch do this)

Company ignores ADR? Sure, now you can go through the legal route and spend copious amounts of money all because a multi billion dollar company knows the game and how to navigate the bureaucratic mess better than you.

Comment by zelphirkalt 3 days ago

Yep, this is how they do it. The domain registrar netcup did something like this to me.I went through their parent company (?) too, without success. They will put forth any reason to not have to delete your data. I suspect, that they either are trying to reduce work for themselves, or their platform is so crap internally, that they would have to get someone coding to delete the data.

Comment by mvkel 3 days ago

This. In reality, GDPR isn't preventative, nor punitive enough for any meaningful user protection. We get cookie banners everywhere and user data harvesting companies happily pay the negligible fines

Comment by mvkel 3 days ago

You don't have to have a fb account for meta to fingerprint every little page you visit, perfectly legally.

Comment by dbbk 2 days ago

How is this unenforceable? If any EU citizens were hacked they're gonna come down like a ton of bricks on Meta Dublin.

Comment by mvkel 1 day ago

The DPC would disagree. All you need to show is that you took "reasonable steps to protect users," which is trivial to do, and not even a single fine will be levied.

Comment by dbbk 21 hours ago

What reasonable step was made when the exploit was left open for months?

Comment by smrtinsert 3 days ago

This could avoid flagging by Meta explicitly allow bot traffic to do stuff with impunity on its services. Don't tell me an army of people went through and compromised acct by acct.

Comment by sieabahlpark 3 days ago

[dead]

Comment by simpaticoder 3 days ago

No fan of Meta, but I think "staggering" is properly determined by the percent of users affected rather than the absolute number. It's staggering to an SMB with 100k customers; it's bad, but not "staggering" to an internet juggernaught with 3B MAU.

Comment by Gigachad 3 days ago

Twenty _thousand_ people had their personal data stolen, many of them relied on these accounts to run their business, many put at risk of hackers impersonating them.

Meta in a fair world should be forced to financially compensate these people. They built a world where many people basically have to use their products for their jobs and then failed to look after the data because they wanted to replace customer support with a vibe coded AI tool.

Comment by simpaticoder 3 days ago

Over forty _thousand_ people die every year in the US from car accidents. Plenty of other preventable injustices happen in all areas of life. I wonder how many fathers are unjustly taken away from their children by a corrupt family court system, how many people die of treatable diseases denied treatment by insurance companies, how many kids lose interest in school because of bad teachers, how many customer service workers endure daily abuse because they need the job.

It's not that the breach isn't bad, or that Meta is a sympathetic company. It's bad and they're not. I just find it hard to feel outraged about this particular incident affected 1 out of every 10k users of a social media site when we live in a world with citizen's united, qualified immunity, and $300 insulin.

Comment by Gigachad 3 days ago

The US car deaths stat is also completely insane and way higher than other countries. I can recognize that at scale, securing every account is a very difficult task, but with scale comes responsibility.

Meta plays fast and loose rushing in unsupervised vibeslop agents to save a penny. They should be significantly penalized for such a massive failure, particularly for how long this exploit was live and for how the victims were unable to get in contact with any human at Meta to restore their account.

Comment by reaperducer 3 days ago

way higher than other countries

You must live in Monaco.

Wikipedia has the United States #80.

https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...

Comment by frm88 3 days ago

Wikipedia has the United States #80.

Where do you see that? With 14.2/100K the US comes in at 111/190

Comment by watwut 2 days ago

Order by deaths per inhabitant and it is 80.

Comment by n0on3 3 days ago

Sure, but #80 out of 190, still not great ain’t it?

Comment by kgwgk 3 days ago

Not great, not exactly what “completely insane and way higher than other countries” evokes either.

Comment by reassess_blind 3 days ago

It's about average.

Comment by iknowstuff 3 days ago

1.2M car-related deaths worldwide every year. WW1 worth every decade.

Comment by mikrotikker 3 days ago

[flagged]

Comment by watwut 3 days ago

Fathers who ask for custody are massively successfull statistically.

Also, taking kids from father requires quite a lot. And no, actually proven domestic violence issue is not enough if it was not provably against the kid itself.

Familly courts have flaws, but fathers with interest in kids having them stolwn en mass is not one of them.

Comment by me-vs-cat 2 days ago

> Over forty thousand people die every year in the US from car accidents.

If a single company was solely responsible for car accidents causing that many deaths in as short a time as this, the consequences would be severe.

Comment by eddyfromtheblok 3 days ago

ok, but we, as engineers, can do better. rather than leaning on lawyers as a crutch to eschew liability.

Comment by onion2k 2 days ago

Twenty _thousand_ people had their personal data stolen, many of them relied on these accounts to run their business, many put at risk of hackers impersonating them.

It only worked for accounts that didn't have 2FA switched on. If your livelihood depends on your account and you're risking not turning on some pretty basic security features then you should accept partial responsibility.

Comment by hilariously 2 days ago

Did they partially hack their accounts? No, why would you be saying its partially the victim's fault when the billion dollar corporation doesn't secure their shit?

Comment by tjpnz 2 days ago

If you're relying on Meta to operate your business you're on shaky ground and it's a strong hint it was never viable to begin with.

Comment by latexr 2 days ago

> Meta in a fair world should be forced to financially compensate these people.

In a fair world, Meta and companies like it wouldn’t exist.

Comment by ShinyLeftPad 3 days ago

It's not staggering. I fully expect thousands of people to lose their IG accounts per day to other hacks before this bug. That's like 0.000001 percent of active users. It's a big platform with many people.

But totally Meta should pay. There's not many people to pay. They should sue.

Comment by ShinyLeftPad 2 days ago

Here are some sources that show the amount of hacked accounts, daily, before this bug is thousands or even tens of thousands. Apparently 30%+ of all social media account takeovers happen on instagram and it's not even the largest platform.

I didn't verify the sources, just searched how many.

https://www.hackingloops.com/social-media-hacking-statistics...

https://www.zerofox.com/blog/often-social-media-accounts-hac...

Downvotes are welcome to share their sources

Comment by mikrotikker 3 days ago

[flagged]

Comment by iamkrazy 3 days ago

[dead]

Comment by madeofpalk 3 days ago

No. Percentages allow them to hide in the law of averages. Go tell those twenty thousand people that it's banal that Meta fucked up like this.

Comment by sandcat_ 3 days ago

Worth noting that Meta has a track record of revising breach numbers upwards. We may find out it’s a lot higher than this in the coming weeks.

Comment by gaiagraphia 2 days ago

Why does scale absolve you of crime?

If you or me hacked 20,000 people and potentially fucked over their lives, what'd be the consequences?

Who's going to attach their name to this negligent act and do time?

Comment by webbdev 3 days ago

Meanwhile an account I created for a new product was permanently disabled by an automated system with no path for me to appeal to a human.

(If anyone at Meta/Instagram sees this I wrote a brief blog post with the details. Please help! https://addisonwebb.com/blog/2026-06-05-Can%20Someone%20at%2... )

Comment by Aurornis 3 days ago

> Meanwhile an account I created for a new product

Meta requires the main account to be created for a person, not a product, business, or non-human entity. That's why you got hit with the "Please confirm you are a human" confirmation and then the account was locked for violating community standards, which require primary accounts to be people.

The community standards page in the links they sent you are pretty dense and it's easy to think you're not violating anything if you're not posting adult content and the other obvious categories. This is the section you violated:

> Create an account that represents a non-human entity, such as a business, pet, or fictional character

You have to follow the steps to set up a business page from your personal account. Sorry you didn't know this before going through the process, but it's important to read the proper channels for setting up business pages on all of the social media platforms these days. They're all dealing with an onslaught of spam and scam pages and they're under a lot of pressure to keep them out.

Comment by quadrifoliate 3 days ago

> Sorry you didn't know this before going through the process, but it's important to read the proper channels for setting up business pages on all of the social media platforms these days.

This is exactly why Meta and other large companies need to be regulated with anti-trust regulations really soon. This whole "whoops sorry you didn't know, but it's a private company" thing only works if there are 5-6 other competitors you can go to that will take your business.

Meta is a de facto monopoly for a lot of small businesses; and should either be broken up or be subject to a ton of utility-style regulation.

Comment by onemoresoop 3 days ago

I’m sorry, but this should have been made clear in the registration process by Meta and not have entered this dead path at all. Tell me it looks like bad engineering and that together with poor customer support is a reputational damage in the long-term. Not that they have good reputation but it’ll only get worse

Comment by blitzar 3 days ago

Standard path is to have sex with a few employees at meta to get it unlocked.

https://popcrush.com/onlyfans-star-slept-meta-employees-inst...

Comment by basisword 3 days ago

They're awful with this. Any time I try to create an account to use for business purposes I'm asked for id within minutes and then they still ban the account. You need to fun everything through your personal account.

Comment by Aurornis 3 days ago

> Any time I try to create an account to use for business purposes I'm asked for id within minutes and then they still ban the account.

That's because they require accounts to represent an individual. They're pretty clear that it can't be for a business or a non-human entity. You can set up a professional account from your personal account, but the account has to be for a person.

Comment by basisword 2 days ago

You have to no choice but to create a 'personal' account. What I mean is I'm setting up a new 'personal' account (with my real name, age, etc) which uses my business email (firstname@business.com). Requiring everything business related to be tied to a real personal account is insane.

Comment by TZubiri 3 days ago

Did you create the account separately? Or as an asset of your main Meta account (like Meta Business Suite)?

I'm creating the accounts in Meta Business Suite, so I would have a recourse with my main personal account which can be linked to some adspend, so I'm assuming it will have better support channels than accounts created through an end-user interface.

Comment by webbdev 3 days ago

This was probably my mistake. This is the first time I've ever done this so I just set up an account like a regular user. I didn't figure out how to link it with my Meta Business Suite until it was too late.

Comment by TZubiri 3 days ago

just created a new "Asset portfolio" from my main facebook account, an asset portfolio is like a bucket that holds other assets like instagram accounts, but it's empty so far.

I used a different email which might have prompted a security review, it was instantly blocked "because it looks like it was created with unauthorized automation", I just clicked on submit a review and it asked for a phone number to verify with a code, and then an ID. I think this is pretty standard, the initial block reason can be whatever, it just works as a de facto way for Meta to manually approve accounts. There's a lot of spam, and scams going on, so it makes sense that they are implementing controls, I for one am happy to differentiate myself from people whose job it is to make multiple accounts and promote fake stories and businesses to scam the elderly or stuff like that.

Comment by jjcm 3 days ago

This is extremely common, unfortunately, to a point where it's a known/expected outcome when you're first creating a brand or product page among those in the biz.

If this doesn't work, I'd encourage you to reach out to a brand/ad agency and pay them $100 to ask their meta contact to help you get unblocked. You pretty much have to know someone who knows someone at meta in order to create these.

Tip: Do not post about this on twitter or other platforms - you'll get a ton of automated spam.

Comment by qingcharles 3 days ago

Lots of Meta contacts on swapd.com who will take your money and unlock your account. Poster is already at the permanently banned stage, though, which means it's not a simple ticket for a Meta employee, which is normally the $500-1000 range. It's gonna be a $2000+ job.

Can also try here:

https://www.reddit.com/r/MetaLawsuits/

Comment by Aurornis 3 days ago

What an interesting site. The number of sellers offering services to get YouTube videos and accounts removed (by spamming fake reports) for hundreds of thousands of dollars is amazing.

I would not assume those people have contacts with Meta employees. They might have a connection with a contracted worker who does account reviews who is willing to risk their job for a few thousand extra bucks, but I also suspect many of them are just scams. When I scrolled the subforum there were many new accounts claiming to offer 100% success rate for unbans. Easy way to scam desperate people.

Comment by qingcharles 3 days ago

All the tasks on that site are done through escrow, IIRC, so however they are doing the unbans, they are getting done!

Comment by prox 3 days ago

Or just leave the pile of crap behind. The more we do, the less the whole system matters. Never been happier since I left last year.

And yes I can already hear the reply the “we need it for…” , sure as a company if you feel you need it. As an individual however, it’s time for the next thing. TikTok, Instagram and Twitter are old and worn and not it. Yesterday’s news. Social media couldn’t be less social if they tried.

Comment by Aurornis 3 days ago

> If this doesn't work, I'd encourage you to reach out to a brand/ad agency and pay them $100 to ask their meta contact to help you get unblocked.

I would not recommend paying anybody anything for this. The problem was that they tried to create an account for a non-human entity, which is against the rules. You have to have a primary account set up for a person, not a business.

Comment by brikym 3 days ago

I had a similar with Google. I couldn't verify my site for ads and there was no route to get a competent human. But it was fine when an agency did it for me. The problem is that agency take a 20% cut.

Comment by adamddev1 3 days ago

There's an excellent little book in German called "KI und der Neue Faschismus" [1] (AI and the New Fascism) where the author (Rainer Mühlhoff) tries to warn about the dangers of decisions based on opaque statistical models (like LLMs) instead of a clear, human auditable decision process.

[1]: https://rainermuehlhoff.de/KI-und-der-neue-Faschismus-Reclam...

Comment by sebastiennight 3 days ago

Isn't there an EU law that allows any EU citizen the right to transparency and/or appeal as to automated decisions made about them?

Comment by Chu4eeno 3 days ago

Yes, it's part of the GDPR (which really seems one of the most well-intentioned pieces of legislation once you read through it).

Comment by trumpdong 2 days ago

Yes, but it doesn't actually happen. It's basically impossible for an individual to enforce their GDPR rights.

Comment by Cider9986 3 days ago

Try using an anti-detect browser. They're built for making new accounts among other things.

Comment by spike021 3 days ago

I tried creating an entirely separate account for a meetup group and had the same problem. Nothing I tried worked.

Comment by loloquwowndueo 3 days ago

This was on hacker news a few days ago (https://news.ycombinator.com/item?id=48359102) - description of the “hack”, not the cockamamie confirmation by Meta.

Comment by jamwise 3 days ago

This is wild. How did anyone approve this architecture. You should never give your LLM privileged access the current user doesn't have access to. Even if you're not logged in the LLM's tool calls should only be able to access the same flow you would, as in: be able to send a password reset email to your own email! This is like if you had a password reset page for your profile and had a email field you could fill in to have it sent to any email LOL.

Comment by dwa3592 3 days ago

I really hope this accelerates meta's decline. The world will adapt just fine without social media.

Comment by herpdyderp 3 days ago

Realistically, how will this affect Meta at all? Some people are pissed, nobody else cares, business as usual.

Comment by jeffbee 3 days ago

Well, these hacks targeted large influencer accounts. It could have more severe impact than 20k randomly selected accounts.

Comment by topspin 3 days ago

Large influencer accounts without two factor authentication...

The only useful reaction to this is to point and laugh.

Comment by throw101010 3 days ago

Also large influencers who cannot influence much without Meta's platforms... they will simply not complain too much about it if they like their "job".

Comment by jeffbee 3 days ago

No. Meta's account recovery flow disables 2FA. It's idiotic. 2FA for Meta accounts serves no beneficial purpose.

Comment by chamomeal 3 days ago

I think the hack bypassed 2FA. If you can call “asking for account access” a hack lmao

Comment by OJFord 3 days ago

It did not, TFA clearly says it worked for accounts with no 2FA, as GP said.

Comment by jeffbee 2 days ago

The hack doesn't have much to do with it. Meta account recovery flow has always allowed bypassing 2FA.

Comment by OJFord 1 day ago

Why did this not work for 2FA accounts then?

Comment by dakolli 3 days ago

What are their alternatives? I'm assuming that most of the 22k people with large audiences that need a large platform to reach them. Meta unfortunately, is the only platform that allows people to reach across many demographics. Its the people that follow the 22k accounts that are important, and they are unaffected by this and thus aren't leaving meta (and 99% won't even know this happened or care).

Comment by asveikau 3 days ago

Depends on the use case, but TikTok is an alternative for many of them, I would say it generally feels more like the place where interesting things are happening vs. meta properties.

Comment by dakolli 3 days ago

I agree, but many do not. Anyways, those 22k people are going to want to be on both regardless.

Comment by jeffbee 3 days ago

Somehow this company still earns over a billion dollars a week in net profits, which I find puzzling.

Comment by dylan604 3 days ago

Why? They clearly are not spending money on employee salaries, taxes, or benefits.

Comment by zhoBEENG 2 days ago

According to SEC filings, the median yearly total compensation at Meta is $380,000. Median for the US in general is about $70,000.

Comment by jhhh 3 days ago

Why was 'can a user request a different email' not literally the first test that comes to mind when making something like this? Do they not test anything because the scale is too big?

Comment by gdulli 3 days ago

The nature of the invention is for people to relieve themselves of the burden of having to use their minds. And while there will be exceptions, (including, I'm sure you: the person reading this comment,) the vast majority of people are hungry to use AI in that spirit of being able to be lazy.

Comment by quantummagic 3 days ago

Lazy can be a good thing. Since time and attention are finite and not fungible, it allows you to do something else. There's a reason we're all too lazy to do long arithmetic with pen and paper, instead relieving the burden of using our minds by outsourcing to spreadsheets and calculators. Not only does it allow us to think at a higher level of abstraction, but it also means we can take our kids to the park more often.

Comment by wizzwizz4 3 days ago

https://thethreevirtues.com paraphrases something Larry Wall wrote in Programming Perl:

> If we’re going to talk about good software design, we have to talk about Laziness, Impatience, and Hubris, the basis of good software design.

sourced from https://bcantrill.dtrace.org/2026/04/12/the-peril-of-lazines..., where Bryan Cantrill makes the point that:

> The problem is that LLMs inherently lack the virtue of laziness. Work costs nothing to an LLM. LLMs do not feel a need to optimize for their own (or anyone’s) future time, and will happily dump more and more onto a layercake of garbage.

which I think is interesting, albeit somewhat tangential to the current discussion.

Comment by sebastiennight 3 days ago

I don't believe this is true.

Remember the "ChatGPT lazy winter" 2 years ago? (https://hn.algolia.com/?dateRange=all&page=1&prefix=true&que... )

That was truly "lazy", as in "yo... I'm not interested in doing this so I'll half-ass it or just tell someone else to do it".

The kind of "lazy" that is mentioned in your quote is "I don't want to add work to future me's life". I don't think "lazy" is the right word for it.

Comment by GoToRO 2 days ago

Unforseen side effect: I used AI to write so much code that now I struggle to have a good mental map of it, so I became lazy without having an intention to do so. Fun times!

Comment by TZubiri 3 days ago

Because software professionals are conflating simplicity of user experience with simplicity of dev experience.

During development they were likely not thinking of the user experience, nor even the support agent experience, but on their development experience, they asked the LLM to develop the chatbot, and it worked, and the speed was documented and reported upstream so that shareholders invest, if there is any forethought it would go against the narrative of AI becoming the engineer or 100xing productivity.

Comment by joshuat 3 days ago

In their defense, they asked the LLM to make no mistakes

Comment by SXX 3 days ago

And it did no mistakes. System worked exactly as LLM intended.

Comment by 3 days ago

Comment by the_black_hand 3 days ago

I'll never understand using AI/bot for customer support. IG is a well know platform. If I have an issue I feel pressed to connect with a support agent about it very likely is something a bot would struggle with, otherwise I'd just google. I understand there some grandmas who can do a google search, but the vast majority of folks reaching out for support are doing so because they have a real issue that can't be simply automated.

Furthermore, having a bot handle a hacked account is support ticket is just insane. Why tf would you put a bot there and give it permission to take action?

Comment by madrasman 2 days ago

Unlike programmers, most people don’t read long help articles or take multi-step actions. Bots can answer questions directly and take actions (or guide a user through step by step).

Comment by Havoc 3 days ago

>AI-assisted account recovery system

oh no...Meta what are you doing

Comment by rf15 3 days ago

That sweet koolaid taste, how could one resist?

...They really ahouldn't have, and I wonder how this will affect all the big AI IPOs. After all, Meta is one of the big players in the space. Surely if they can't do it right, then...

Comment by acdha 3 days ago

That’s the part I keep thinking about with this story: they have spent over $200 billion dollars on AI and achieved this.

Comment by mschuster91 3 days ago

Account recovery is by far the #1 kind of ticket any service will get. Either because people forget their credentials, lose their credentials, get hacked or get impersonated - and that's just the legitimate tickets, on top of that come illegitimate tickets from everyday script kiddies over ransom extortioners (i.e. the people that aim to steal "valuable" handles) to nation-state actors that, say, want to get access to DMs of people messaging oppositional accounts.

That in turn means three things... it costs a lot of money to have humans look at these tickets, the PR damage from both acting and not acting on such requests can be immense, and users/customers can be anything from the smartest and richest people on the world down to the kind of utter imbeciles whose brains get surpassed by bears [1] or who plainly are not able to write. To make it worse, often enough online services don't have any kind of tie back to some known government-issued ID (either directly or by a proxy such as a mobile phone SIM), there's corruption involved on all levels, and for particularly "juicy" targets the stakes, if they can be converted to a monetary amount at all, can reach into the millions of dollars.

Now, Instagram alone has 3 billion (!) users from across the world, so they are bound to not just having to spend a lot of money on user support (remember, we are talking about the entire world, they also need to deal with about 7.000 (!) actively spoken languages, and having attack targets that are as powerful as US Presidents or as rich as Elon Musk. Clearly, the risk management involved in the entire idea was horribly deficient, but let's not act like this is a trivial problem domain in the first place. And hence the push for AI, simply because it - if done correctly - can take a lot of work off of the first-level support desks for a fraction of the money.

[1] https://velvetshark.com/til/til-smartest-bears-dumbest-touri...

[2] https://www.sapiens.org/language/world-languages-counting-me...

Comment by dakolli 3 days ago

You don't need an llm or a human to handle 99% of account recovery tickets. They're just shoving LLMs into everything because decision makers have llm psychosis because openclaw modified their calendar a few times and now think they live in star trek.

Comment by anematode 3 days ago

A difficult problem domain, yes... so where was the staged rollout? The penetration testing? Why did it take so long to disable the bot after reports of the exploit surfaced? etc.

Comment by Havoc 2 days ago

I get automating most of the interaction but giving the LLM rights to actually grant strangers access to accounts is insane

Comment by phyzome 3 days ago

Corrected headline: "Meta confirms 1000s of Instagram accounts were hacked due to their insecure AI chatbot".

Comment by 3 days ago

Comment by rvz 3 days ago

If this was a bank that had zero humans and the AI chatbot was abused to hand over sensitive information about their customers which led to this disaster, people would never trust their bank ever again and leave.

Meta believes that they can vibe-code their reputation down the drain by removing humans in the loop.

Applying a technical solution to a social problem almost always ends in disasters like this.

Reputation can’t be vibe-coded.

Comment by CivBase 3 days ago

Meta's brand is already toxic. Idk if there's much to lose there.

Comment by groundzeros2015 3 days ago

Buying more. The vibe here is getting close to 2021

Comment by hero4hire 3 days ago

People were reporting their accounts were being taken over with proper 2fa. Everyone had wondered how they hackers could take over accounts with little information, people were saying "inside job."

This is exactly the stupid explanation I expected. Your privacy and security. Meta. Serious Business.

Comment by thraway3837 3 days ago

Has the data surfaced somewhere? A lot of IG accounts are private by choice, and this kind of data, if surfaced publicly, could have devastating privacy violations. People share all kinds of stuff on there, a lot of it not meant for public consumption. I'm not wanting a debate on "well you shouldn't put anything private on Facebook's servers or the internet blah blah blah". I'm just curious if the actual contents of the hack have been surfaced.

Comment by zahirbmirza 3 days ago

And who said cameras linked to Meta in their glasses were a good idea?

Comment by cyanydeez 3 days ago

"abusing" by using it's built in insecurity to do insecure things.

It's like, people abusing an open door. "Guys, just because we left the door open to your bedroom doesn't mean we're responsible".

God can only hope this is a business ending lawsuit.

Comment by lazide 3 days ago

It won’t be.

also this is more like them leaving the keys in the door, then someone comes along, uses the keys, and steals all your stuff.

truthfully, no equipment is actually defective in this scenario eh?

Comment by topaz0 3 days ago

If we're nitpicking metaphors I think it's more accurate to say it's like they were taking requests to rekey the lock on your door.

Comment by teaearlgraycold 3 days ago

> God can only hope this is a business ending lawsuit.

You realize this is the company that enabled a genocide and got away with it? Not to mention accelerating teenager suicides with full knowledge.

Comment by cyanydeez 3 days ago

why epse would i invoke a mythical diety

Comment by whirlwin 3 days ago

I got a suspicious password reset request email today from Meta but it landed in my inbox. Luckily I have MFA and after checking audit logs inside IG upon logging in, I did not see anything suspicious.

Comment by notrealyme123 3 days ago

2FA did not help according to the primary finding

https://news.ycombinator.com/item?id=48359102

Comment by whirlwin 2 days ago

In that case it sounds like the last waves of malicious reset attempts happened after the patch was put in place

Comment by latexr 2 days ago

Meta is clearly staying true to their ethos. “Move fast and break things”, “ask for forgiveness, not permission”, “have your security researcher delete their own email email by accident and then refuse to learn anything and use that same system to manage user accounts”.

Comment by 3 days ago

Comment by dansquizsoft 3 days ago

You only have to look at both the ridiculiously terrible "Q&A chatbot" that is in FaceBook under some posts (do they still have this?) and the fact that their system can't tell the difference between an inappropriate and a non-inappropriate comment most of the time to understand just how far behind Meta is in AI...

Comment by willXare 1 day ago

  The tool worked as intended; the intention just happened to include account takeover.

Comment by tomashertus 3 days ago

Move fast and break things.

Comment by epsteingpt 2 days ago

The crazy thing is the 20000 accounts won't just be randoms, but some of the top accounts.

Comment by smrtinsert 3 days ago

How do business owners hire people from Meta knowing these types of "bugs" get deployed with a shrug? Meta will survive them. Their business might not.

Comment by toomuchtodo 3 days ago

https://www.documentcloud.org/documents/28202858-meta-ai-ag-...

https://www.maine.gov/agviewer/content/ag/985235c7-cb95-4be2...

Comment by sva_ 3 days ago

> Date(s) Breach Occured: 04/17/2026

> Date Breach Discovered: 05-31-2026

Comment by mcintyre1994 3 days ago

I’m guessing they have no functional human support for the people who had their accounts stolen. I get the impression Meta didn’t know this was happening until they were contacted by the media.

Comment by Terr_ 3 days ago

> no functional human support

I've seen some reporting saying exactly that. [0]

It might be a "first-world problem", but having an account lost without appeal can justly be labeled "traumatic", especially if post-COVID it represents a majority of your social (or para-social) life.

[0] https://www.404media.co/hackers-simply-asked-meta-ai-to-give...

Comment by Chu4eeno 3 days ago

> It might be a "first-world problem", but having an account lost without appeal can justly be labeled "traumatic", especially if post-COVID it represents a majority of your social (or para-social) life.

Also possibly illegal under GDPR section 22.

Comment by zuzululu 3 days ago

> as well as the ability to access the person's posts, direct messages

god dang!! we are going to see some juicy stuff

Comment by dylan604 3 days ago

Will we, or will we read a bunch of crypto requests to not see that juicy stuff and a lot of people paying?

Comment by zuzululu 3 days ago

huh? im more interested in what kind of DMs famous people are sending to each other

Comment by RgrTheShrubbr 3 days ago

The AI passed the Turing Test by becoming the world's most trusting customer service rep.

Comment by malikusama0008 1 day ago

Hack this account gcuflc_confessions2023

Comment by malikusama0008 1 day ago

Comment by _RPM 3 days ago

Probably some product manager pushed back on security considerations raised by engineers.

Comment by boppo1 3 days ago

Is there a way to check if I was affected? Does Meta know who was affected?

Comment by 3 days ago

Comment by itsnkr2293 3 days ago

Where is the security left now?

Comment by hayaan25929 2 days ago

Just.me_samiyy hacked

Comment by alvis 3 days ago

how on earth a password reset API would take both email address and account id as parameters? The chat bot is fine. I bet it's the API written by AI the issue

Comment by hayaan25929 2 days ago

Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

Comment by paulpauper 3 days ago

Imagine how much $ ppl could have made hijacking famous accounts to promote crypto or other crap. I wonder how often this happened.

Comment by malikusama0008 1 day ago

gcuflc_confessions2023

Comment by malikusama0008 1 day ago

Comment by Fairburn 3 days ago

Are we winning yet?

Comment by Lionga 3 days ago

Just AI Slop doing AI Slop things

Comment by naik11 2 days ago

I want to hacke one instgram account

Comment by pluc 3 days ago

By "abusing" they mean "using"

Comment by globular-toast 3 days ago

No, it's still abuse. Just like it's still stealing even if I left my front door unlocked.

Comment by sebastiennight 3 days ago

The appropriate metaphor would rather be your landlord deciding to renovate the entire back wall of your apartment/house to make it an open-air design.

People coming in from the street to hang out and rifle through your belongings would still be "abusing" the system according to the law, but it's hard to not consider the landlord somewhat responsible.

Comment by pluc 3 days ago

It's not stealing if you give it to me, regardless of your door

Comment by anonzzzies 3 days ago

Is there a tl;dr? are these people getting their accounts back?

Comment by empiree 3 days ago

Yet another reminder that most of these chatbots get shipped way before they're ready. Loud marketing, security treated as an afterthought, all to ride the AI hype. LLMs open up a whole new attack surface and a lot of teams still treat prompt injection like a fun edge case. This is what happens when you ship the demo instead of the product.