Terence Tao: Mathematics in the Age of AI [pdf]

Spotted the German guy. ;)

AI is a tool that reshapes knowledge work but the ultimate goal remains about value.

If you read about life in the 19th century, it was more turbulent than today with change.

We seem to frame these debates against some random week in spring 2017 that if it ever existed at all, was extremely fleeting.

I think the bigger issue is that social media doesn't exactly provide historical context. If anything it provides a type of ahistoric , hysterical context to the present moment.

Then if you have no social media, everyone feels more and more like a hysterical lunatic living in a reality that doesn't exist if you just go outside and put the phone away for 5 minutes.

In the same way children still learn their times tables even though calculators can do the job better.

To me (and most normal people) knowledge pursuit is about inspiring others with discovery and the intellectual joy of doing something yourself.

I disagree, for a lot of people knowledge pursuit is about the knowledge and the results of that knowledge. IE the end result is what matters.

As a simple example, Pythagoras's theorum is useful to lots of people all over the world as in the actual equation. I'm not sure most for most people that the knowledge that Pythagoras himself went through an intellectual journey to discover the theorum is more important than the equation itself.

If we use a thought experiment to say that instead of Pythagoras discovering the theorum it was actually discovered by artificial intelligence do you think most people would care? I would say not.

I think a lot of people like the idea of being at the forefront of their field and their intellectual contributions being able to push the frontier foreward. If AI simply gets good enough to largely absorb the frontier they don't find as much fun in rediscovering what AI has already discovered. People forget however, this has already happened to many fields, chess being one example.

Whether AI gets there with maths is an open question but to say AI has no value in maths whatsoever is to say that the practical implications of a better understanding of maths that AI could potentially give us is worth nothing.

I often hear that people are "uninterested in what AI has to offer" but I don't really understand it. I understand this is a mix of ethics and engineering that I don't often engage with but I do not understand why someone would explicitly state that they are not interested in improvements to the world based on the source of the improvement.

The comment I am replying to did not say that LLMs cannot create improvements. That would be a different discussion! It instead says that it does not care about fruits born from the LLM tree.

It seems highly likely to me that through a combination of LLMs and traditional optimization software like vizier that manufacturing, industrial processes, scientific research, or even bakeries will see some improved output through the mixing of domain specific expertise and enablement from LLMs.

I think right now, in the extreme hype environment we live in where people say AI will do everything and take over the world and take your job and give you nothing, people are burned out. Once we get past this point through I am hoping people start talking about what LLMs can do that will actually improve our lives and which can actually work. Much of what people tell me LLMs can do, don't actually work reliably. But there are many cases where LLMs have been a step function in capabilities.

Examples: OCR that really works, fuzzy search, fixing scoped bugs in software, building prototypes of software, etc.

They think they are better than you and me and everyone is just a human resource who has no right to enjoy anything.

That is why they hope for the industrialization of the sciences. The C students can feel equal to the A students.

Why Tao goes along? He is Chinese and they have an insatiable social pressure to be the first and best at "new" things. Since Yau tried to rip off Perelman, my view of the Chinese pressure cooker influence on Western academics has changed.

Especially since the results aren't the same as in the times when everything was more relaxed in academia and the stopwatch idiots hadn't taken over yet.

Your best interest only means your current best interest. What makes for the best form of personal interest is evolutionary, not static.

Dresden Codak as a story has a concept of "Dark Science" where the world has the benefit doesn't come from publishing science, but from utilizing it. If you understand a piece of science when nobody else does, and integrate it into your technology, you have a strategic non-academic physical advantage. Trade secrets basically. And trade secrets used to be the norm prior to patent law. We could be in the pre-"AI patent" era where the government created incentive to share your AI findings isn't made strong enough yet.

Science is today built on the shoulders of giants, but what if an agent swarm makes your own tower of giants in a day? Why do you need an ecosystem of peer review, if you can discover your own science from first principles without meaningful effort? Yes, it's duplicated effort, but why does that matter if the monetary cost of effort trends to zero? The value of peer review is going to have to contend deeply with itself just like how open source developers have had to contend with the value and mechanisms of inbound AI generated PRs the last six months.

(Disclaimer: I had quite a bit of formal training in mathematics at one point, but have worked in software for more than two decades now. I wasn't on the applied side, but I recall that MATLAB evoked similar concerns back in the day.)

> It goes entirely against the culture of research mathematics to hoard secrets. If you've done something interesting, you want to share it, and it's in your best interest to share it.

You're describing an enviable human culture with a reward system suited to people. People publish papers etc. to enrich the human domain and to get recognition. AIs don't care about things - and neither, I'm asserting, will their owners. That culture will become a vestige.

Mathematics as the underpinning of science and technology that can be exploited for inter-state competition, including cryptography but not limited to that (image analysis, targeting algorithms, conflict modelling, whatever) won't see the light of day. The AIs that discover the mathematics will be paid for by the warfighting budgets of states, and by the corporations that feed on them.

If you can derive some special formula that prints money from accurate prediction of asset valuations if and only if you're the only one with the efficiency, you're incentivized to keep it quiet.

If you're a nation state that's cracked a simple method to decrypt modern "pre-quantum" encryption schemes with less compute/wall time using a development in mathematics, it is in your interest to keep this quiet so that the parties you surveil to not switch or adapt. Similarly if you've figured out a handy new form of encryption, you're incentivized to prevent leaking of details as it is harder to reverse such operations without details of the methodology.

Or you've designed some special optimizer that you then bundle in a subscription service. Your business now only profits while the algorithm is secret. e.g. The DVD Copy Control Association's CSS encryption was only a valuable thing that the company could charge for or entice clients with before DeCSS was published/became popular.

Update: see also the later slide where he problematizes corporate secrets in relation to math culture.

Everyone should just read the slides, really

This is the crux of everything you've written and I strongly disagree. The purpose of knowledge is the sheer power it brings, and if there's factory scale churning out of it, I'll gladly take it.

As a closer analogy, consider this probability (I'll be stealing from the game Stellaris here): let's say a few hundred years later, when we are spacefaring, we discover a vast cache/library of knowledge of an alien civilization, we won't just pass up on exploiting it because we need to "enjoy the process of acquiring it on our own" or "derive the satisfaction of sharing the discovery with others". No, I and I daresay most others will take as much as we can without restraint in a quest to make all our lives better/more enjoyable.

And if that's ok to do, using machine intelligence to mine/grab any new knowledge or new math as Tao is suggesting here should also be acceptable and welcomed.

TT has real results which is why his opinion is worth listening to even when you disagree. These random accounts are dismissable immediately.

Except to make jokes about of course.