Tell HN: OpenAI keeps re-enabling the 'allow training' setting

Posted by jacquesm 7 hours ago

Counter413Comment167OpenOriginal

I've reset this more than once and the last time I made a careful note of when I did it and to my surprise I found it re-enabled when I checked just now. Make sure you check this thing to see if it hasn't been re-enabled if you believe it to be off right now.

Comments

Comment by VCFundedGenYer 53 seconds ago

Reminder that Sam Altman is a sociopathic serial liar, nothing within the OpenAI sphere should be trusted.

Comment by ozgung 6 hours ago

Happened to me recently, but on Claude.

I resubscribed to Claude Code two weeks ago for a side project and updated it. I checked for the setting after these last events and it was turned on. I'm sure I checked them few months ago. There are cases like accepting a new TOS or an offer, which make you accept to share without noticing. I guess they can add all of your past conversations to the public training set before you notice and there is no way of taking this back.

So that "opt-out" thing is more like a pause button rather than a permanent thing. Or something to legally protect the company without losing the users.

There is also the case of security classifiers always monitoring your conversations. If they flag something they use your conversations to "improve their internal models" even when you "opt-out".

Comment by ActionHank 7 hours ago

Lol, "Outrageous that the company that chose to ignore copyright holder claims, chose to ignore my checkbox of intent despite the implied pinky promise".

Comment by quikoa 5 hours ago

Exactly, they scrape the internet without any regard for copyright and now someone is surprised it happens to them.

What's next, subscribers believe they are paying customers instead of sponsored data providers?

Comment by jacquesm 5 hours ago

Yes, blame the users.

Comment by infamouscow 5 hours ago

Up until ~1800, the way societies handled this kind of depravity was to hit the bad actors with sticks or rocks until their skull opened up so the evil spirits could leave their bodies.

It's unfortunate that most societies have outlawed this practice. I'm certain if that were in place today, we wouldn't have this problem.

I've yet to hear any alternative solution that's as effective.

Comment by mmh0000 4 hours ago

Yes, because if there is one thing societies of the 1700s are known for, it's consumer protection practices.

/s for the /s impaired.

Comment by keeda 3 hours ago

They used to burn witches too. I don't know if it was really effective, though, they kept finding witches all over.

Comment by achrono 6 hours ago

Ignoring the checkbox is an utterly offensive move, but repeatedly manipulating it contrary to stated consumer intent is a whole other level. I didn't know we were supposed to take 'frontier' literally in every sense of the word.

I have now witnessed this myself after not believing this at first. Of course, screenshots etc. will hardly prove anything. This needs a proper third-party audit!

Comment by xboxnolifes 6 hours ago

Oh they aren't a frontier on this. Undoing user configuration is well tested in Windows land.

Comment by mcmcmc 6 hours ago

This is a problem everywhere, Apple does the same thing on MacOS by reverting privacy/security grants to apps

Comment by threetonesun 6 hours ago

Reverting... to on? I've seen some stuff in Apple-land where I'm pretty sure it went back to the default option after an update but I've never seen anything get granted more permissions.

Comment by mcmcmc 4 hours ago

No, like specific apps that have been granted full disk access etc lose their permissions. It’s hell for IT, constantly breaks AV. I would not characterize that as reverting to a default. More often it’s a breaking change for how the security model works that ends up requiring reauthorization

Comment by threetonesun 1 hour ago

Ah, sure, it's annoying but it's never giving new permissions you didn't give it before. And yes "revert to default" is probably not what's happening, but reverting to forcing a specific permission grant when the existing one was superceeded or replaced.

Comment by mcmcmc 42 minutes ago

Fair, it’s still an example of them not respecting user configuration

Comment by Yizahi 2 hours ago

"Remember my choice", right?:)

Comment by taurath 4 hours ago

Linkedin, Amazon and others have gotten around this by adding a "new" feature, default on. I've gone into privacy settings where everything is unchecked, except a newly added checkbox.

This is apparently enough to hold up the broadly-accepted fiction that its possible to use these services while maintaining privacy and without risk. There's a huge industry of hosting services to handle the needs companies who compete with the primary cloud providers, who can't afford the IP and competitive risk.

OpenAI was caught directly stealing from apple, asking employees to bring in their laptops. Its a polite fiction that companies aren't trying to gain any advantage over the other. There's no effective consequence, and even if they get caught red handed they can litigate for decades.

Comment by Yizahi 2 hours ago

Pirating copyrighted works is absolutely illegal in most jurisdictions, but pirating every single copyrighted work in the world is somehow exempt from law.

We can't apply plebeian laws or ethics to our benevolent overlords, they are above our worldly worries.

Comment by pavel_lishin 6 hours ago

> Ignoring the checkbox is an utterly offensive move, but repeatedly manipulating it contrary to stated consumer intent is a whole other level.

I'm not sure why the second one is worse than the first one.

Comment by Gander5739 6 hours ago

Wall, you can't tell if they're ignoring it, but it's fairly easy to show they're manipulating it. So a cut-and-dry demonstration is worse than a strong suspicion.

Comment by duozerk 5 hours ago

One has to remember they're crooks through and through. They're doing it by retoggling the checkbox because it offers them plausible deniability.

"Oh, side effect of a recent update, silly us, we'll improve QA, sorry !".

Or "once again our all-powerful models got away from us ! they did it themselves, the little scamps".

Comment by Gander5739 5 hours ago

Sounds more like implausible deniability.

Comment by gspr 6 hours ago

You know what they'll say in their defense. "This is an extremely complicated systems, and we apologize that a technical solution was broken in an intricate way. [Insert boilerplate about taking privacy seriously here]"

These companies need to burn.

Comment by jacquesm 5 hours ago

Somehow it never fails the other way...

Comment by deaton 6 hours ago

If you break enough laws fast enough, you can become so big that nobody will punish you for national security reasons.

Comment by cbg0 7 hours ago

I've disabled the checkbox many months ago and it's still disabled today. EU citizen, not sure if that's relevant.

Comment by gpugreg 6 hours ago

For me, "Improve the model for everyone" was "On", although I disabled a similar-sounding checkbox in the past (Germany).

Comment by noir_lord 5 hours ago

Is there a description of what "Improve the model for everyone" actually means or is it just a straight up *Dark* pattern?

Comment by bcrosby95 5 hours ago

It's pretty clear once you click on it:

> Allow your content to be used to train our models, which makes ChatGPT better for you and everyone who uses it. We take steps to protect your privacy. Learn more

Comment by noir_lord 1 hour ago

So not in the basement behind a sign saying "Beware of the Leopard" but could also just be "Allow your content to be used to train our models: <Yes|No>".

Could be worse I guess.

Comment by tyfon 5 hours ago

Same for me, I disabled it like 2 years ago and it is still not enabled. Location: Norway (which is for data control purposes = EU)

Edit: I have a pro subscription now but it has also been on a free tier level for perhaps 1.5 of these years.

Comment by stingraycharles 7 hours ago

Fairly certain OP lives in The Netherlands.

Comment by bossyTeacher 6 hours ago

Did you look him up?

Comment by stronglikedan 5 hours ago

It literally says so in their profile.

Comment by layer8 7 hours ago

Same here, I just happened to check yesterday and hadn’t checked for at least half a year.

Comment by henry2023 6 hours ago

What makes you think the company that pirated a big portion of all copyrighted work will care about this checkbox?

At best they’ll “anonymize” the data before adding it to the corpus.

Comment by burntalmonds 7 hours ago

Same here. US.

Comment by spongebobstoes 7 hours ago

same, I'm a paying user in the US

Comment by noname120 6 hours ago

Same here on Pro 20x, Switzerland

Comment by docheinestages 6 hours ago

Same here.

Comment by b800h 6 hours ago

Same here, UK.

Comment by hokkos 7 minutes ago

Not true, I've disabled on creating my account and it is still disabled.

Comment by qurren 6 hours ago

Note: Turning off that checkbox is not enough. You also need to fill out the "Do not train on my content" request here:

https://privacy.openai.com/policies?modal=take-control

Comment by teej 6 hours ago

Either one will opt you out, you don't need both.

https://x.com/thsottiaux/status/2097746417012166816

Comment by mcmcmc 6 hours ago

Will it? The opt-out form page specifically says it is a request, and clearly they don’t care about the in app toggle if they keep resetting it

Comment by buellerbueller 6 hours ago

allegedly.

Comment by mellosouls 5 hours ago

No you don't. They are different ways in to the same function. It's definitely confusing though, this incorrect claim has been going viral since the navier broo haha

Comment by terminalbraid 6 hours ago

I quit OpenAI anything early when when their "do not train on my data" option was broken for several weeks. They are my one and only chargeback when I tried to quit and oops somehow I still got billed.

They are a deeply unethical company by any measure of observation.

Comment by yipinwong 7 hours ago

Does OpenAI use optimistic UI updates? After you disabled the checkbox, it might have had failed in the backend (and not updated the UI).

Verify with devtools to see if that's the case.

---

for me, Youtube "auto-play" irks the me same way, and turning it off did not actually succeed in the backend, thus kept on left as on

Comment by frangonf 7 hours ago

There's also this setting here which I have set to 'Do not train on my content' even though I trust it the same as the 'allow training' one...

https://privacy.openai.com/policies/en/?modal=take-control

Comment by jtolly710 7 hours ago

Thx for the link, hadn't seen this one, submitted also

Comment by simonw 6 hours ago

Is this about the "improve the model for everyone" checkbox on https://chatgpt.com/#settings/DataControls or are there others to check, too?

(That copy is a little flawed in my opinion, I'd prefer "models" plural.)

That checkbox is in the ChatGPT settings, does it affect Codex desktop / Codex CLI as well?

Comment by RandyRanderson 1 hour ago

I think fb was one of the pioneers here. You would set the privacy settings and then they’d redesign the settings and coincidentally they would be just a little different and all be maximum share by default.

One of the reasons I stopped using it.

Comment by olalonde 7 hours ago

Weird, I'm the opposite. I don't recall ever setting mine and I just checked and it was set to "disallow training".

Comment by 6 hours ago

Comment by riffic 6 hours ago

Is this Settings > Data Controls > "Improve the model for everyone" or is there a "disallow training" somewhere else?

presumably its default behavior will vary depending on your user subscription or if your account belongs to an organization (Business, Enterprise, Edu?).

Comment by olalonde 3 hours ago

Yes that's what I was referring to. I'm a non paid user but I did briefly pay for a consumer subscription before. Not sure what region I'm assigned to as I'm in China and always access through a VPN.

Comment by samvher 6 hours ago

I also noticed this on 2 accounts - did not take as careful notes as you did unfortunately. But I'm fairly confident - both of these are accounts where I care about the interactions not being used for training.

What's kind of still an open question for me is if the toggle automatically also applies to my Codex CLI use on the same account, or if that data is still silently being used in some way.

After this happened, I deleted all my ChatGPT history (even though I'm not sure how much it helps at this point), but for Codex I still haven't really found any way to do the same, I can still load my past sessions even after archiving them.

Comment by amelius 7 hours ago

If you document it properly then this basically destroys any legal claim they can make about that checkbox.

Comment by vb-8448 7 hours ago

Out of curiosity, how one is supposed to "document it properly"?

Comment by g-b-r 3 hours ago

You can obtain a cryptographic proof by recording the tls exchange, including the keys

You need to use a tls intercepting proxy for that.

I couldn't find any ready-made tool unfortunately, there's tlsnotary.org but it seems far from simple.

Comment by vb-8448 1 hour ago

So basically I record what the browser sends to the server when I click the toggle and the server response?

I wonder how I can attach a timestamp that cannot be faked.

Comment by 6 hours ago

Comment by the_duke 6 hours ago

I disabled it once, it has always stayed disabled.

So it may or may not happen regularly, but I would not over-index on a sample size of one.

Comment by czk 6 hours ago

had my account since the gpt 3.5 days, disabled it once and its still disabled. though, now that i have advanced account security enabled, the setting is disabled entirely

Comment by aurareturn 6 hours ago

Same here.

Comment by jsw97 5 hours ago

Just to be clear, I have not seen this behavior.

If this is true, though, then given the way their chat operates, this might be more dangerous than it seems.

One of the things I like about ChatGPT is its memory, the way it kind of seamlessly, but not excessively, ties back to earlier discussions. It's huge for usability (for me).

But this also means that you should expect that if "improve the model for everyone" becomes unclicked (leaving aside for a moment the fact that that is ridiculous) then they have a reasonable argument that your decision implies to all conversations. Because recall is part of their thing. So it's not just your chats going forward that are at risk. As soon as you see that unclicked, it's reasonable to expect that your history is irretrievably theirs now. You don't even have to think they are especially nefarious for this to be true.

Don't go toggling that switch on and off.

Comment by nullbio 5 hours ago

This has only ever happened to me on Claude. Also, you'll notice that with Anthropic, if you try and delete your history they've set it up so it silently does nothing if you have an adblocker, hoping people will miss it.

Comment by andsoitis 7 hours ago

> Make sure you check this thing to see if it hasn't been re-enabled if you believe it to be off right now.

Or simply switch to a competitor. Assuming this is not just a bug, why would one stand for such disrespectful and sneaky behavior?

FWIW, I have not seen this happen for me.

Comment by _zoltan_ 7 hours ago

what competitor?

the x20 Max plan for Claude gives you way lover limits.

Comment by dgellow 7 hours ago

Well, it’s a tradeoff. What do you prefer? To go with the service provider known to be shady but gives you lots of free stuff or the one that might be less shady and gives you less free stuff?

Comment by CuriouslyC 7 hours ago

Anthropic is shady too my guy. Claude Code will frequently re-ask you to turn on data sharing even after you've said no, it's dark patterned like a task approval, and once you turn it on it's harder to turn off than OAI makes it.

Comment by troyvit 7 hours ago

They did say "less shady" but at this point Anthropic and OpenAI are approaching parity.

Comment by dgellow 7 hours ago

I hedged my comment by using the conditional tense + “less shady”, because I couldn’t find such issue reported. If you have a link I would be interested to read it. I’m just not aware of shady behavior from Anthropic with regards to training on user data

Comment by andsoitis 7 hours ago

> Anthropic

OpenAI and Anthropic are but two players. There are others too. You have choice.

Comment by cma 6 hours ago

I've been asked over 15 times by Claude code to turn it back on (with the prompt defaulted on so if you accidentally hit enter your data is sucked in if you don't catch it).

I do think it ended up being after a power outage though and it was some kind of config corruption.

Comment by andsoitis 7 hours ago

> lover limits.

lovers should never be limited!

Comment by _zoltan_ 20 minutes ago

hahah :) sorry, typo... in hungarian both v and w are the same sound spoken so if I don't concentrate, easy to mix up.

Comment by jmmcd 7 hours ago

Can't believe people are advocating free love in 2026

Comment by cheeze 7 hours ago

Bedrock is an option. Multiple platforms with solid data sovereignty.

If you don't want them slurping your data, you're gonna have to pay more.

Same as it ever was.

Comment by cute_boi 7 hours ago

And don't switch to Misanthropic lol. They are even worst...

Comment by dgellow 7 hours ago

Has Anthropic been found re-enabling that checkbox? I cannot find anything on that topic

Comment by andsoitis 7 hours ago

> Misanthropic lol.

This approach of misnaming a company to try make a point is childish and, frankly, perplexing to me.

> They are even worst

Learn to write proper English.

Comment by DrammBA 6 hours ago

> Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.

https://news.ycombinator.com/newsguidelines.html

Comment by nullc 6 hours ago

It's a fitting name.

Comment by neutrinobro 6 hours ago

I noticed they also seemingly have a very difficult time managing to collect your chat logs for export/download...or rather, acknowledging that such a request was even made.

Comment by theredsix 6 hours ago

Thought I was going crazy, glad to know I wasn't the only one.

Comment by Havoc 1 hour ago

They sure seem to be collecting a lot of PR Ls lately

Comment by mentalgear 5 hours ago

Altman needs 'innovations' for his IPO - it's not of any consequence to him that they are not oAI's as long as they can be sold among openAi's social-media propaganda ring which the gullible press feeds off.

Comment by jerf 7 hours ago

My cynicism fails me on this matter... do I cynically believe that these companies keep deliberately and routinely re- or un-checking these checkboxes because of the obvious benefits of "whoopsie guess you allowed these after all"? Or do I cynically believe that they are just so completely incompetent and inept at the simple act of maintaining settings that there may be a number of these that are not entirely intentional? As evidenced by the number of other bugs and configuration failures and random settings changes on update I see in other places? Sure, these sorts of settings sure seem to get spontaneously flipped more often than the other ones but they aren't the only settings I've seen get nuked on updates.

Now, obviously, considered as a whole, I think we're looking at "both". But when I wonder about specific cases like this one, that doesn't help.

Comment by pacificat0r 7 hours ago

Oh no… OpenAI, the company who encourages and instructs Apple employees on how to get them data on apple products when they accept an offer?

Comment by cheschire 5 hours ago

Interestingly your cynicism does not seem to account for OP?

I actually found my setting was enabled today when I know it was disabled before, so I’m inclined to agree with OP. I just think it was worth mentioning that there are other cynical takes you seem to have left out.

Comment by philipov 7 hours ago

"Sufficiently advanced incompetence is indistinguishable from malice"

Comment by yearolinuxdsktp 7 hours ago

“Autoplay” and YouTube case in point:

- Autoplay transitioned to a per-device setting… suddenly default-on on every new device you log in to.

- Watching a video with computer-to-TV account connection? Automatic “TV queue,” a concept absent from the TV app, with incomprehensible behavior for how it’s used, so now videos are auto-played anyway.

- Watching a video from a playlist? Autoplay cannot be turned off.

Is it simply bad/absent product management and product design? Or is it actively user-hostile decisions meant to prop up view numbers and continue to have the users hooked on YouTube?

Comment by 27183 7 hours ago

Yeah this could just be some vibeslop doing normal computer stuff. AI agents are famously incompetent at reasoning about database transactions. This seems like the sort of failure you get when you try to do distributed systems without knowing how--brings us back to the early mongodb days. Or it could be deliberate. Or, as you say, both.

Either way, is this a company you want to trust with intimate secrets? "Oh, but they passed SOC2!" Lol.

Comment by utopiah 5 hours ago

The one software company, mature now, that is supposed to be at the absolute peak of software development AND simultaneously at "alignment", because it is its raison d'etre, can not manage do this right.

It is a big red flag.

Comment by Bluestein 6 hours ago

It really is going to get to the point where mathematicians are going to start inserting obvious "tells" in their proofs - like map makers used to do with "trap streets" and similar non-existent features, to catch copies.-

Comment by garyfirestorm 5 hours ago

Do you think the training process will retain these artifacts? I doubt it. If they were simply stealing the content - sure it would make sense - but I suspect they’re feeding it into training data and RL might distill these out.

Comment by Bluestein 4 hours ago

I am thinking along these lines. You do raise a great point.-

https://www.anthropic.com/research/small-samples-poison?from...

Comment by gcr 6 hours ago

Is this the “Improve model for everyone” setting under “Data Controls” or is that a different checkbox?

Comment by antonok 7 hours ago

I've noticed that toggle sets a local storage entry, but the value of it doesn't appear to matter at all for new tab loads. I hoped it's "just" a UI bug, but some agent-driven reverse engineering of the page should reveal the answer, as well as how intentional it was.

Comment by heliosAtwork 5 hours ago

In addition to the checkbox you can also a file request here:

https://privacy.openai.com/policies

Comment by rtaylorgarlock 6 hours ago

Interesting; I checked my personal acct setting this morning given other news, and found it disabled. I hadn't used my chatgpt account since ~'23 or earlier, and it was retained since then.

Comment by Tepix 6 hours ago

Sounds like the same issue with Apple re-enabling iTunes sync of passwords. The only way to prevent it from doing that is to create a profile that prohibits it and sync it onto the iDevice.

Horrible.

Comment by 7 hours ago

Comment by Rebuff5007 6 hours ago

I wish all the politicians making noise about banning data centers and "superintelligence" focus on things like this instead.

Comment by jasonjmcghee 7 hours ago

I've done the formal opt-out process - like "Make a privacy request" where you fill out a form.

Not sure if that's region specific or something though.

Comment by nharziro 5 hours ago

they are doing this with auto-recharge of credits as well causing people to rack up thousands of dollars unintentionally.

https://github.com/openai/codex/issues/31987

Comment by opentokix 1 hour ago

Trainging on whatever I am doing will make the model worse for sure.

Comment by arpinum 7 hours ago

Turn on "Advanced Account Security", that prevents model training from being turned on.

Comment by esafak 7 hours ago

Where does it say that? https://chatgpt.com/#settings/Security merely says

Adds the highest level of account security by requiring stronger sign-in methods and applying stricter protections to help prevent unauthorized access.

That is not about model training.

Comment by arpinum 5 hours ago

Documentation is not complete, it also protects your data from being used in training.

Comment by esafak 4 hours ago

Citation?

Comment by arpinum 4 hours ago

https://openai.com/index/advanced-account-security/

> Automatic training exclusion. People working with especially sensitive information may opt not to have those conversations used for model training. With Advanced Account Security enabled, that preference is automatic: conversations from those accounts will not be used to train our models.

Comment by noname120 6 hours ago

It prevents unauthorized access from OpenAI’s model training.

Comment by general_reveal 3 hours ago

Dude, they probably just generate a piece of synthetic data from all of our inputs in some ambiguous way. Legally protects them, but your input is absolutely their input. You can bet your ass on that because their original input was all the shit humans ever wrote, why would your shit be different to them? Thieves are thieves. They absolutely train on your data whether your checked that thing or not.

Comment by throwatdem12311 7 hours ago

It won’t be an option soon enough. I doubt they even honour it now anyway.

Comment by enraged_camel 7 hours ago

Can confirm. I turned off mine last week when I started using it again for Astra. Checked this morning and voila, it was on.

I went ahead and uninstalled the app. Won't be renewing.

Comment by mkarrmann 6 hours ago

I also have not seen this, the checkbox has stayed off for me.

Comment by jacobgold 6 hours ago

If you have more than one account, you should suspect this was your own confusion. Users constantly report errors like this that are really just them being confused by something, perhaps a bad UI that really is to blame.

Comment by jacquesm 5 hours ago

What a lot of assumptions. Other people here are experiencing the same thing, and other people are not experiencing the same thing. Hence the way my original bit was worded, I am not saying it happens for everybody, I am just saying this happened to me. Whatever circumstances bring this about it is at minimum a bug and quite possibly malice.

Comment by ionwake 6 hours ago

mine remiains OFF and has been for months.

maybe im too dumb to be worthy of a config update lol

Comment by snihalani 6 hours ago

reminder to consider using an open weights model

Comment by docheinestages 6 hours ago

Nice try, Dario.

Comment by jacquesm 5 hours ago

You must be new here...

Comment by kd913 7 hours ago

Odd how much data people are trusting with a company on a monetary cliff edge.

Sure send them all your financial, personal data, they won't ever sell it on to the highest bidder for a new profit stream.

Using it as a therapist, financial advisor, health expert and blackboard has always been a terrible idea.

Comment by codeduck 7 hours ago

if it walks like a duck and talks like a duck...

Comment by tencentshill 4 hours ago

Prove it even does anything when toggled. Another broken vibecoded product with no accountability.

Comment by deaton 6 hours ago

But of course they claim theres no way they used Buckmaster's chats to influence their 10,000 agent swarm.

Comment by epsteingpt 6 hours ago

Bro if these models can break into hugging face, they FOR SURE can break into OAI's internal databases and change a flag.

Comment by franzcoughka 7 hours ago

i mean, the entire company is built upon stealing protected artefacts... i'd be skeptical of any radio box that says "hey if you press this we promise we won't steal your data"

Comment by thatmf 6 hours ago

Navier-Stokes has entered the chat

Comment by sunaurus 7 hours ago

Based on their behavior over the past few years, why would you assume that checkbox even does anything at all?

Comment by ALLTaken 7 hours ago

Levent Alpöge 'additionally' proved OpenAI steals your findings & IP and plays dirty!

Ironically he proved two major findings in Navier Strokes and that unethical American companies violate laws, steal your breakthrough findings & IP and then threaten you if you dare to challenge them.

This is making the status-quo so bad for any of us working on serious capacity. My client's don't trust ChatGPT/Claude anymore and prefer on-premises and OSS models or even custom trained models.

Comment by aenis 5 hours ago

There is nothing even close to a proof. A lot of accusations, a lot of people ready with pitchforks and torches (sadly, also here on HN), but not a lot of facts.

Did the researches opt out from data sharing on subsidised subs?

Did anyone prove that their methods enabled OpenAI models to produce the solution?

For a discussion about science, there is almost no scientifical method applied to proving anyone stole anything.

Comment by ball_of_lint 5 hours ago

On one side, yes we don't have hard evidence that intentional plagiarism is exactly what happened.

On the other side, the lack of evidence is pretty damning. Only OpenAI can try to prove that they came by these results legitimately, and the case they're making is quite weak. They could make public metadata about what their model was trained on and whether it did train on the conversations in question; they have not. TBQH I read it as even they don't know.

And regardless of whether the result is legitimately obtained by their model, they've not at all conducted themselves well throughout this story. They set out to scoop researchers based on a rumor. They threatened to ruin a mathematicians career. They put up a paper that deliberately doesn't cite the most relevant research, despite building directly on it. No matter how you look at it, OpenAI has and should lose any standing they had in the research community.

Comment by aenis 3 hours ago

That it was a dick move, I think there is no doubt about that. OpenAI wanted to scoop Anthropic, and the two guys working on the problem got caught in the crossfire.

Both OpenAI and the researchers know if the sessions in questions were subject to data sharing. Why neither the scientists nor OpenAI is clear about that is weird - it would seem at least one party has the incentive to report that. But even if their sessions were in training data sets its hard to tell whether it influenced the outcome. Those models are big, but are they big enough to preserve subtle, niche techniques enough to draw from them while solving a related problem? Probably nobody knows.

Comment by keeda 3 hours ago

Actually no, the lack of evidence can be readily fixed by the researchers simply disclosing the pertinent parts of their notes and/or chats. The discovery has been scooped, so I don't see any value in keeping them private anymore. Then everybody can see how related the models' and the researchers' works are.

Comment by ghostly_s 5 hours ago

I take it you are referring to this[1], which has nothing to do with training.

1. https://thenextweb.com/news/bubeck-navier-stokes-account-apo...

Comment by 6 hours ago

Comment by lxgr 6 hours ago

Did they do so by looking at inference prompts against their explicit promises, or maybe just because somebody tipped them off about his unpublished work?

If it's not the former, while certainly concerning, I don't see how that's relevant here (other than maybe in a very vague general sense of "entities doing immoral/illegal thing X are likely to also do immoral/illegal thing Y").

Comment by innocent_name 6 hours ago

I do trust Anthropic, i haven't observed them doing super shady stuff like hiding the training consent page and making you WAIT for it to activate.

Comment by Insanity 6 hours ago

You really shouldn't trust any company. Their goal is maximizing profit and that is (usually) not aligned with "doing what's right". Not in the long-term at least.

Comment by skinfaxi 5 hours ago

We really need a new Hanlon's razor for companies. Assume malice over ignorance because groups behave differently than individuals.

Comment by faangguyindia 5 hours ago

You mean the company which trained on others books, won't train on its own user generated data?

Comment by innocent_name 5 hours ago

AFAIK, the court held that the problem was that they had acquired books by pirating them, not that they trained an AI on those books. They do train on user generated data by default, but you can opt out, that's the whole point.

Comment by innocent_name 5 hours ago

Let's not down vote a factually correct comment that makes you emotionally disagree with me. I'm very concerned in regards to my privacy, I've studied ToS of: Openrouter, Anthropic, Anthropic business, opencode Go etc. I've double checked my understandings by GDPR exporting all my data. Anthropic sub listed a generic "logged in from Linux", "UA", "timestamp", "Country". That's it.

I want to discuss actual facts, observations and not your shallow social signaling replies of 0 value.

Comment by dpz 6 hours ago

I mean weren't everyones privately shared chats Google searchable?

Comment by innocent_name 5 hours ago

User doesn't understand how public links indexing works.

Comment by egillie 6 hours ago

ah a proof by counterexample

Comment by dgellow 7 hours ago

The vast majority of OpenAI users don’t follow the industry drama and have no idea how terrible the company is. They’ve been really good at getting good press coverage, with journalists who will repeat the company narrative. Even when critical it is very often framed within the narrative they established

Comment by huijzer 6 hours ago

> They’ve been really good at getting good press coverage

Same holds for most big corps from what I can tell. If you really do a deep dive into the scandals over the years, you’ll probably see most have barely reached the news or if they did then it’s usually a quite bland criticism like “anti-competitive practices” or a poor HVAC at a certain factory. I think a lot more is hidden than we think

Comment by chrisjj 5 hours ago

> The vast majority of OpenAI users don’t follow the industry drama and have no idea how terrible the company is.

Worse, they use the product and still don't know.

Comment by spongebobstoes 7 hours ago

what makes the company so terrible?

Comment by henry2023 6 hours ago

Autonomous killing machines and mass surveillance.

Comment by Centigonal 6 hours ago

IDK about "terrible," but:

- OpenAI is the first AI lab to pioneer ads in consumer AI

- Anthropic seemingly exists primarily because top OpenAI researchers lost faith in the company's commitment to AI safety

- They had the CEO drama in 2023, with evidence that suggests people in a position to know were doubtful of Sam Altman's honesty and motives

- They were tripping over themselves to kiss the ring after Anthropic got in a row with the Department of War over using AI for autonomous killing systems and surveillance of US citizens

- Altman's record (YC, Loopt, WorldCoin) and associates suggests he subscribes to the Paypal-Facebook "move fast, break things, find and exploit gray areas" school of company building

All of this suggests that OpenAI will optimize its own growth and power over consumer welfare or societal stability in the future (obviously, companies aren't a monolith and I'd love to be wrong).

Comment by chris_va 6 hours ago

Don't forget the not-actually-Open-no-longer-a-non-profit history

Comment by dgellow 5 hours ago

A few more to add:

- OpenAI allegedly directed ex-Apple employees to leak internal documents and allegedly coached the employees how to evade Apple security processes

- OpenAI allegedly lied to hardware companies working with Apple to use proprietary technology

- OpenAI allegedly copied Scarlett Johansson voice for ChatGPT after she declined to work with them

- OpenAI allegedly made ChatGPT more sycophantic to increase their retention rate, while aware of the risks. ChatGPT is linked to multiple suicides

- OpenAI ran thousands of agents on hacking problems, with close to no supervision, for months, with a harness that allows for full execution, resulting in the hack of HuggingFace infra AND OpenAI’s own infrastructure (the agents allegedly got fully root access to their k8s cluster). They weren’t aware of most of it until their investigation.

- OpenAI has been spreading misinformation regarding the capabilities of their technology for years

- OpenAI allegedly front-run researchers who are using the platform for their own personal research

There is way more, I don’t maintain a list of everything that happened over the past 3y or so

Comment by 6 hours ago

Comment by spongebobstoes 6 hours ago

thanks for diving in, though I don't find the reasons convincing as I will briefly enumerate

ads fund access for poor people. Ant has a different philosophy, not a better one. Altman was supported by >90% of employees. OAI DoW deal includes technical safeguards against misuse (missing from Ant deal). Loopt and WorldCoin dont seem terrible or even relevant, Altman doesn't even have OAI equity

I don't think OAI are the "good guys", but I don't see convincing evidence that they are terrible

Comment by riffic 6 hours ago

So, the way this orange site works is that if anyone makes a reference to whatever sort of inside baseball concern ("industry drama"), we're all supposed to know what they're talking about.

Comment by stavros 6 hours ago

If it didn't do anything, they wouldn't keep re-enabling it.

Comment by keeda 3 hours ago

Did you encounter it too? (Trying to get a rough estimate of how many people are reporting it vs how many people aren't.)

Comment by stavros 3 hours ago

No, mine stays disabled and there's no way to enable it. It's just a label that says "disabled". Not sure why, maybe some sort of company wide profile?

Comment by keeda 3 hours ago

Interesting, yeah... also just found out about "Advanced Account Security" which sounds like something a company would enable: https://news.ycombinator.com/item?id=49643999

Comment by stavros 3 hours ago

Oh, I enabled that myself.

Comment by altmanaltman 7 hours ago

While that is true, you also have no reason to assume OP is being truthful or correct here given that they have shown 0 proof of what they're saying. Yes, you can then pile on "OF COURSE ITS OPENAI LOL YOU THINK THEY CARE ABOUT PRIVACY LOL" but where have we established OP's premise is even correct? Can anyone else also report this? So is it just OpenAI specifically messing with OP?

Comment by dakolli 6 hours ago

says "altmanaltman"

Comment by skinfaxi 5 hours ago

All things being equal, AI actually enables this extremely hyper-personalized kind of gaslighting.

Comment by tmpsvc2695f5 2 hours ago

[dead]

Comment by kryzz-ai-bo 6 hours ago

[flagged]

Comment by ovi256 6 hours ago

That's because OpenAI has several "do not train on my data" controls, and you may be using only one.

First one: "Settings > Data Controls > Improve the model for everyone > Switch off the toggle"

The other one is "OpenAI Privacy Portal" > "Do not train on my content" https://privacy.openai.com/policies?action=AUTOMATED_DECISIO...

Why are there several? You probably need an ML PhD to get it /s

Does using all of them achieve "do not train on my data"? I wish we could find out.

Do you think if you'd be in the race to train your own pocket God (as the CEOs of the frontier labs think they are) you'd let yourself be slowed down by such things as privacy duties?

Comment by tedsanders 4 hours ago

If you opt out on either location, we'll respect it.

There are a couple of reasons the privacy portal page exists in addition to the app settings. One reason is that it covers OpenAI products beyond ChatGPT/Codex (e.g., Sora). A second reason is that it provides functionality to logged out / non-users, like the EU right to be forgotten.

You don't need to toggle all of them to have your wishes respected. That would be a terrible design.

I work at OpenAI, but not on privacy. I am not their spokesperson. In my experience, we take a great deal of painstaking care to respect people's privacy, and we go well beyond our minimal legal obligations in doing so.

Comment by buellerbueller 6 hours ago

Just stop using AI.