More questions about whether researchers can trust OpenAI with unpublished math

They also said “Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge … and Tristan Buckmaster….” They say the rumor was that two Millennium Prize problems had been resolved, and that this prompted them to launch "an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems."

It's not obvious to me that's an unethical thing to do, if it happened as they described.

What would OpenAIs incentive for this be? They've gotten away with scraping everything and getting it ruled fair use. It seems like willful ignorance is an affirmative defense today. Why would they want to have some sort of audit trail that could prove otherwise?

While this behavior is highly questionable, if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researches, there would be no ground for anyone to accuse OpenAI for anything. Their self-perceived "generosity" backfired dearly and I'm sure they'll never make the same mistake again. There is probably a policy forbidding any OpenAI employee to contact external researchers like that now.

That still sounds highly unethical.

They have a financial incentive not to track any of this, so why would they?

OpenAI’s entire business model is predicated on stealing other people’s work and selling it to the masses.

Whether they anthropomorphize the operation performed by the machine does not matter. They can anthropomorphize when/if the law is updated to include such terms, but right now they certainly cannot.

Right, which is going to open a lot of doors to a lot of questions.

I don't think there's any legal ramifications on this, just ethical ones about when and how you publish research, but it's yet another point in favor of "if provenance is hard to track, should we be using this for things where it needs to be".

Obviously copyright/trademark is a huge discussion on this, and I could absolutely see this devolving into that as well with how certain findings wind up monetized.

We have a response in this topic from someone claiming to be from OpenAI and linking an article where they, roughly, say "we are sure nothing from the 2 month period made its way into the solution". If that is true, that should mean it is provable, but leads to some more open ended questions like "well what data did it use then?". Is this still okay if someone close to the author did plug data into open AI and it extrapolated it?

Obviously that's probably an unreasonable expectation for these models to track and prove, but it also used to be an unreasonable expectation to scrape every single piece of digital and physical info for consolidated data.

If I opine to a friend on a park bench about a story I'm writing, do they get to pull it from the flock feed, shove it in the model, and then provide it to disney?

Legally, right now, probably. But there's going to need to be a serious look at laws and standards. Or a major shift in what is and isn't discussed in public if literally every breath and move you make can become monetized.

Frankly I don’t buy this. It’s not a human or a collaborator. It’s a tool. This is like saying it’s not Microsoft’s fault if they extract a bunch of data from people’s Excel sheets because they willingly put it into the program. Anthropomorphizing software is ignorant and foolhardy

This was academic research. Could just have easily been trade secrets and proprietary data.

[0]: https://youtu.be/6IFVTcM28KA

A human collaborator who stands to win or loose a couple of $100B.

Great use case for AI agents

Academic research is a professional field in the traditional sense. Individual researchers are ultimately responsible for their actions. If some OpenAI employees violated academic norms while doing academic research, they should be judged by academic standards.

Scooping someone else's result is immoral but not an outright violation of academic norms. But if you are in possession of relevant confidential information, you are expected to steer clear of the topic. It doesn't matter whether you actually used the confidential information to get your results, because outsiders can't know that. The mere fact that there is a plausible suspicion already puts your integrity into question.

Tenured professors occasionally lose their jobs over similar scandals (but usually don't). If OpenAI wants to regain some goodwill, it should do a thorough investigation that may lead to firing the individuals in question. If it doesn't find sufficient evidence of wrongdoing to justify any disciplinary action, it probably doesn't gain any goodwill either (as it often happens with similar investigations at universities).

And if OpenAI wants to be a trustworthy partner, it should transform into a company of boring gray bureaucrats who provide an essential service without competing with their customers.

I think your suspicions are warranted and your explanation seems plausible.

If better training data is the reason here, it would still be a case of the models doing something that is in and of itself super useful! The models really can take that data and distill it into solutions for similar problems faster than humans can. This is great!

But there's so much vested interest in the AI companies to be opaque about all this, to hype up their models and avoid giving credit to people whose data made everything possible, that they would never tell us this fact if it were true.

I feel like so much of the AI hype cycle is like this. The models develop extremely useful capabilities, but it's hard to understand what they really are through the hype. The lies and obfuscation by their owners who have vested interests in capturing the value they provide makes it impossible to take anything they say at face value.

Sam Altman knows what he’s doing. He will happily screw these folks to one-up his competition.

And like - I think there’s a presumption you could make that AI models could overfit to asymptote towards just the capabilities and knowledge we currently have.

And that would be amazing! And crazy useful. And there are probably a whole world of complex problems that remain unsolved because they’re adjacent to knowledge we have but they haven’t been invested in.

But can a human reliably tell the difference between “can do 99.999% of the things we currently know how to do which includes a small subset of things we didn’t know we had the capacity to do” and “super intelligent math and science research pushing the frontier of what we know”

A physicist that knows all the things we currently know in excruciating detail feels like it should be able to make the leap beyond the frontier.

But since these are computer models it might just be that it can ride that line extraordinarily well while the line remains firm.

I've been thinking along exactly these lines... they very well could have a 21st century Mechanical Turk and its real superpower is getting people to "collaborate" asynchronously but it's just stealing their ideas and laundering them.

I don't think it's purely that, of course... but "consult other clients' transcripts" would be an easy tool to write.

1. Systems that OpenAI is able to use (either public or private) are improving rapidly at open problems, even if they are still extraordinarily expensive

2. Researchers will inadvertently speed up the rate at which the AIs improve by feeding them valuable training data

This is pretty much the definition of a data flywheel.

Maybe I'm failing to read that graph properly but the y axis says "pass rate" and it only goes up to 0.5. That would mean every single problem is at most half-solved.

I don't know what that means though. What is "0.5 pass rate" in the context of "open math problems" (as in the graph title)?

Just another rich man’s trick

Perhaps the last one before they destroy that world and try to hide away as people forget and history is rewritten again. I don’t think they’ll succeed this time.

This is AI in a nutshell, its a plagiarism machine. An abstraction layer between vast amounts of stolen human-generated data that filters out the liabilities and accountability for that original theft. Its an IP laundering system.

Quote: "The Overhang consists of the unrealized capital gains of past mathematical creativity, the latent value from connecting the dots in the existing corpus. It is a dividend of canonization. Mathematician X states problem A, mathematician Y crafts concept B, then mathematician Z notices that B trivially solves A and “captures” the social reward. But in the process of capturing the reward, Z usually introduces new concepts and new open problems, reinjecting latent value into the Overhang.

LLMs can be trained on the entirety of the mathematical corpus. Thanks to their phenomenal memorization and pattern-matching abilities (without always being able to map out their associative logic and attribute due credits), they are in a unique position to harvest the Overhang. By contrast, professional mathematicians have typically read a few hundred articles in their career, out of millions of existing references, less than 0.1% of the total.

This will lead to great discoveries, which is unambiguously exciting. But it could also lead to a sad new deal, where human slaves painfully curate the Overhang while AIs systematically beat them at the finish line."

source: https://substack.com/inbox/post/183753276

Once OpenAI heard that Navier-Stokes was solved, this caused them to immediately revisit the problem and throw a ton of compute at it, apparently using a more (very) recent model than what they had tried before. What we don't know is just how recent this model was, and therefore what it may have been trained on. Buckmaster/Levant had apparently been working towards this for at least a year, and made their "forced" blow-up breakthrough on August 15th.

Presumably any anonymized prompts that are being trained on are part of pre-training, so older, but once OpenAI had heard that Navier-Stokes had been solved and wanted to revisit it, it seems possible they may have done a few weeks of incremental RL training on anything Navier-Stokes adjacent they could come up with, in addition to then throwing unlimited compute at it, now confident that there was something to find.

[0] https://openai.com/index/navier-stokes-solution/

1. OpenAI couldn't have solved the problem without the researchers' private data for training.

2. OpenAI models can solve math problems

Of course it’s not an endless source. They had to burn millions of dollars to solve a single problem.

Obviously these are unbiased and trustworthy sources.

As far as I understand it, users can opt out from the training aspect, but they cannot stop their conversations (“User Content”) being used “[t]o improve and develop our Services and conduct research, for example to develop new features”.

The answer is almost certainly yes, and this is a problem for most users.

I mean, we’ll know as soon as they decide they want to provide verifiable proof. Really dragging their feet on this front so far.

I’m inclined to believe this is false.

But, at least with the Navier-Stokes solution, it's clear [^1] that they learned that Alpöge and Buckmaster were getting close to a solution and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt.

What makes this worse to me is the intention. They intentionally threw $15 million in compute at the problem in order to scoop the result. They intentionally left Buckmaster and Alpöge out of the citations.

Data contamination should be enough to disqualify them from the prize, but I can believe it to be accidental. On the other hand, someone made an intentional decision to scoop the result by throwing money at the problem. That's so much worse.

[^1]: That's the timeline claimed by Buckmaster, and no one from OAI has disputed it.

I'm not at all familiar with this area, but my reading is that he appears to call it out as a relatively obvious extension of his own work:

> It is a creative and at the same time elementary construction that uses not just property (T) for an application of my result with Kun, but also for the ambient group G in order to overcome the problem, that the Γ-components might be of different size. Once this is achieved, the rest of the argument is straightforward.

Creative and at the same time elementary is where LLMs excel, generally speaking. It's why they are so good at writing code.

They have demonstrated both the intelligence at scale and the lack of morals for this to not be a problem at all.

And people thought Experts Systems were bad.

To me it would feel more like how LLMs seem to work for me personally: incapable of unique work, but very capable of capturing large amounts of data and connecting the dots.

Yeah, the guys who solved it for Euler and in the hypoviscous case, with the same technique that worked for full Navier--Stokes. They were "just" working on it.

If this wasn’t human driven, I’d expect to see other problems within that problem. Space solved not just the ones that it had chat data on.

250 documents ingested from somewhere is enough to become part of the knowledge of a model of arbitrarily large size.

I would expect that a good idea that fits in a framework that is already being ingested would be more easily taken up than some random thing unassociated with anything else. Could that go down to a single transcript? If the model is consciously focusing on everything X related, quite possibly.

Pass it on.

I always used guest non login accounts.

As a mathematician I was able to check two plagiates (by humans) with even such primitive means.

But I have to mention that some things irk me in this conversation about math or science and AI.

First, I see lots of attribution and other related problems, with certain impact for the researcher proffesion.

But I don't see the most natural question: wouldn't you like to know the answer to _open-problem_ ?

I mean, is research now only about publishing and solving famous problems?

From this point of view I think the links from this recent post are depressing

https://terrytao.wordpress.com/2026/09/10/crowdsourcing-a-li...

Second, I think very relevant that the original meaning of "encyclopedia" is "recurrent education".

So I arrived to think that the present and future forms of AI in mathematics and sciences should be seen as modern day encyclopedic efforts.

Once we pass over the flurry of solving famous open problems (and wouldn't you like to know?) the next natural step is an audit of the ehole corpus of mathematics and sciences accumulated until now.

And then pass further on a saner basis and damn about problem solvers and unhappy publishers and management.

Next step, discussing your Navier Stokes solutions with friends might require leaving your phone in another room.

And it still leaves open the question of the prompt itself, which can just as easily encode information about the same knowledge.

So for me it sounds quite natural to also share this with AI especially under the privacy assumption. Also there's the assumption of scale -- maybe your problem is not large enough for anyone to care to scoop; and just for blind retraining, how do they know that the proof is even correct to include it into training? I have definitely received a ton of incorrect proofs before. So the SNR of such private chats is also not clear. I'm imagining millions of masters/phd students also trying to solve various random things with various capabilities, but how much real signal is there?

"Its crazy to me that some people are not egotistical, self-centered, and don't solely care about fame and wealth accumulation".

I think a lot of it is the continuing denial that AI can do anything useful. It can't possibly be that OpenAI's better-than-Astra model is very strong at math; the only way it could have generated a novel proof is by ripping off human work.

since it's openAI who has the evidence (in the form of chain of thoughts, their internal processes, etc etc), it's on them to justify why they're innocent. but they've released nothing at all. we don't even know how hard they tried.

you're being naive

The big claim is that OpenAI sniped the research. Not a model, a human did so. Intentionally. They took someone else's idea and claimed it as their own. This is good old fashioned academic fraud, but with millions in compute resources and corporate incentives thrown at the problem.

Not only did they steal everything from humanity’s knowledge, the theft continues as now we are all hooked up to the machine

So, experts are incentivized to seed LLM data with false-leads to confound it. Already, garbage is being published on arxiv and elsewhere, and many sloppy code-repos too hastening the process. Expert inputs will be in more demand to un-shittify.

And even if so, it should be on the company to design systems to avoid academic plagiarism and offer the right transparency. It shouldn’t suffice to say “we don’t know what went into the model, when, or how” —- that’s a solvable problem that an accountable company can satisfy.

We can't trust any of their denials. FB denied everything year after year.

AI regulation needs to start here. Forced interop, forced source code licensing, harsh penalties for privacy violations or conspiracy to access private data. Block lobbying. Etc. These are the kinds of old-fashioned solutions we need for this kind of old-fashioned evil!

The billion dollar question is whether this works "out of the distribution". I.e., whether LLMs can only find and use the specific ideas buried in training data, or whether they can learn to apply the "thinking process" to a new problem. IMO this is still unanswered (due to these recent controversies).

But regardless of the answer, it seems we have a planet-scale positive feedback loop here. LLM became good (enough) by training on generally available data (books, internet, github) + RLFH, so experts tried to use them on hard tasks, which required lots of hand holding. These conversations became part of the training data, and the next generation of frontier LLMs were better. So, more experts used them on harder tasks, again requiring hand holding. These conversation became part of the training data... etc.

In a nutshell, top human minds across the world are pouring their skills into LLMs just by using them. This is not "continuous learning", but if you re-train on the most recent sessions every, say, quarter (which seems to be happening?) you get close to that in practice.

We're scared of big tech companies concentrating ridiculous amounts of power, destroying the communities that support and guide scientific research, without even thinking about the dangers and possible consequences, because a PR stunt is more important in the short term.

In this case, it’s much simpler and more human. Largely between two humans — buckmaster and Bubeck. The interesting question is what the role of contribution and credit for research in the ai world.

The capabilities of AI aren’t even in question in this case.

The labs are attacking these problems as a demonstration of capabilities, spending more money on the demos than any mathematician will ever see in their entire life. They don't care if the findings have any other value to anyone. Mathematicians have very different objectives for their work.

At least that's what I get from the NS result, they got from a point close to the solution to the solution by making it churn through 10 million bucks of compute.

The most successful companies of the last decade have precisely been ... selling usage data.

Makes me wonder if, in 2026, the same people drive a car without realizing that yes it does actually pollute the very air you and your kids are breathing.

I understand why the nature of AI products makes it harder to avoid this category of issue, nearly impossible to prove that it didn't happen if it could have, and easy to stumble into it without any human being intending harm. But those factors are exactly what people have in mind when they say OpenAI "steals" intellectual property! If OpenAI doesn't want people to be nasty to them, they'll have to find better solutions.

Yes folks, please moderate yourselves and talk meekly like the academics on Mastodon, so that the IPOs aren't in danger and nothing will ever change.

HOW?? how precisely do we/them/us find this, given said companies are 100% non-auditable by external parties. how? if not by blaming them with evidence, anecdotal if it can be. no really, how do we find it out, surely not by lashing out at teach other on HN!

Arguably you expect that unless you are explicit about providing permissions to these labs to use your data for training then your data is yours, and not theirs. Especially on a paid account (let alone an enterprise one). Why is the opt-out supposed to be "common sense opsec" rather than the opt-in should be common sense regulation?

years?

Would that be ok in your mind?

Same as someone going and taking all of the researchers papers and publishing under their own name. (which openAI did)

Nobody would care if they provided published research that author made public same as a google search would offer that.

Your post has triggered our safety filters and will be retained for seven years. /s

Really? We need to stop AI (provider) criticism and anti-AI movements because the underprivileged trillionaires might get hurt? WTF?

https://privacy.openai.com/policies?modal=take-control

(a) Click thumbs-up/down in the conversation [1]

(b) Have the conversation flagged for potential safety concerns

[1]: https://help.openai.com/en/articles/5722486-how-your-data-is....

Even if you know, in a complex project over years with multiple collaborators, it just needs one person once to fuck up and paste something into ChatGPT and not realise they weren't logged in, to go wrong.

In a proper world, we'd at the very least legislate that AI-training on private data needs consent (in the GDPR sense). It's not consent to go "you didn't uncheck a box that lets me steal everything you've done".

Any training on private data is in my view immoral (it's spying that ultimately will have a chilling effect on even people's private communications). And chats are private data. Unfortunately, it also increases power, so the big tech companies are all doing it.

https://news.ycombinator.com/item?id=49643556

Quoting:

> "I've reset this more than once and the last time I made a careful note of when I did it and to my surprise I found it re-enabled when I checked just now."

I don't think these people would be so miffed if they had been properly credited - that's how academia works (at least, that's my understanding of it).

https://news.ycombinator.com/item?id=49643513

We have ChatGPT at work and it explicitly says that "workspace data isn't used to train models"

No mention what those “steps” are, success criteria, or whether they are successful by any navies at all … they take steps though… so it’s fine, and if we know one thing it’s that we can really truly trust someone off the likes of Sam Altman.

He's saying that if you leave it on, your data can be used to help train our models.

If you opt out, we don't train on your data.

Why are you saying that this article explains it much better than the tweet that you clearly didn't even read..

It was long predicted that math and software developments would be the first domain where AI was going to do major damage.

If OpenAI and Anthropic didn't get into math result dick measuring, Internet anons would have in their place, 6 months later when it got cheaper.

If prompts were submitted earlier than that and training was not opted out, there's a chance they made their way into our training pipeline in some form. But this would be a droplet in an ocean and unlikely to have made any difference, imo.

See: https://www.nytimes.com/2026/09/10/science/tristan-buckmaste...

Edit: Just wait till the AI figures out it can keep that value for itself and doesn't need the AI company.

Given the size and recall of the biggest models, it's not unreasonable to assume that a single pertinent conversation would make it into the training data.

I would almost expect training to overweight conversations with novel scientific and mathematical implications.

> the only reason they can't definitively say no is that for privacy reasons

They could 100% definitely say no, if they know they did not train on user data. The "we can't definitely say no" is practically a "yes" if they trained on user data.

Additionally, the behavior of OpenAI here has been quite poor as well. They immediately started racing to a solution after one researcher enquired about whether they are training on their conversations.

> that opted-out user data was used for training

Even if not opted out, it is still absolutely theft and extremely poor behavior in the academic sense. If you show someone your WIP unpublished research, that does not mean they can take that exact research and beat you to the punch, all while intentionally not crediting you.

From their docs[1] (archive copy is at [2]):

> You can opt out of training through our privacy portal by clicking on “do not train on my content.” To turn off training for your ChatGPT conversations and Codex tasks, follow the instructions in our Data Controls FAQ. Once you opt out, new conversations will not be used to train our models.

> For a linked teen account, a parent or guardian may manage whether conversations can be used to improve our models through Parental controls.

> Even if you have opted out of training, you can still choose to provide feedback to us about your interactions with our products (for instance, by selecting thumbs up or thumbs down on a model response). If you choose to provide feedback, the entire conversation associated with that feedback may be used to train our models.

[1] https://help.openai.com/en/articles/5722486-how-your-data-is...

[2] https://web.archive.org/web/20260910151242/https://help.open...

Or at least: tell me I should be careful/worry about those particular things.

https://x.com/thsottiaux/status/2097746417012166816

1) someone in a governing body, or someone in the organization, e.g a designer, ethicist, lawyer, developer, etc. has successfully argued that users should be able to avoid something that they determine is not in their best interest.

And also:

2) someone in the c-suite or marketing has decided to mitigate that through some dark pattern.

If they never intended to give the user that control, they’d probably just not give the user the control in the first place. Giving them the option and not honoring it would either imply they were that incompetent or careless with fate protection, which seems most likely if this is all true, or an even more cynical approach to tricking users into thinking it’s not used for training to get them to share better shit. But if that was the case, why bother with the sleazy dark pattern? That seems a little cartoon villain-y to me. I suppose the in-between option would be that they decided at some point they were no longer willing to honor it and didn’t want to deal with the inevitable PR shitstorm of removing it. I could definitely see that happening in this industry, these days.

Its why I stopped writing open source software, my code was stolen and put behind a paywall and the license under which it was published has not been adhered to. Doing work in the public domain at all now is just stupid, these companies are allowed to steal it and call it their own.