We need to pivot math towards intuition and accessibility.
Math has built a gated, inaccessible institution which - by design or not - served as a moat.
I believe math has little to do with mathematical notation - intuition is much more important. One can have intuition but not be able to “read math” - much as many musicians don’t “read music”.
Reading math however does not guarantee ideas or intuition. And those are what math needs.
I think LLMs are the single best things that were invented to make mathematics accessible.
I loved mathematics when I was young and wanted to be good at it, but I got burnt so many times by teachers who could not explain even if their life depended on it.
Asking an LLM and discovering how different theories belong together and what are genuinely open questions, along with different philosophical interpretations has been a wonderful gift.
I have long accepted that Mathematics is a language where you must go "all in" because for the most parts institutions are fundamentally unable to explain it in an intuitive way. There are people who "just get it" and are blind to the struggle of people who need a different approach in learning it. There's something about the field where a generational survivorship bias has created an environment where math is mostly taught to likeminded people, who then don't see why anybody could not see the difficulties learning it.
I know this is not true everywhere, but the only reason I could finish my studies was because there are some genuinely excellent online lectures on youtube, along with massive support from friends who have already been through the ordeal.
If I had had access to an LLM, I could have actually taught myself from scratch and be more playful in learning it. And no, I am not willing to learn lots of math "for fun", I can think of better "for fun" things that actually help my profession and cater to my actual interests and not just some academic ideal.
You would be surprised at the number of people (who study math formally) who just memorize proofs and having no clue what it all fundamentally is about.
> We (humans) should help humanity flourish. I understand that not everyone agrees. I have been called a “speciesist” for being “too human-centric”.
this is insane. the "speciesist" people aside (the best way for all other species to flourish is for humans to eradicate themselves right now, which is completely mad), i have the opposite problem: the _axiom_ seems to be: we should help ME flourish. "Me" as in people who are raking in trillions for their own very special selves right now, at the cost of everyone else's future, while none of them can be trusted to hold my cell phone for a second.
> if an industry commits to the axiom of helping humanity flourish
where does he see such industries, outside of maybe nonprofits?
> Po-Shen Loh (Chinese: 罗博深; pinyin: Luó Bóshēn; born June 18, 1982) is an American mathematician specializing in combinatorics. Loh teaches at Carnegie Mellon University, and from 2014 to 2023 served as the national coach of the United States' International Mathematical Olympiad team. He is the founder of educational websites Expii and Live, and lead developer of contact-tracing app NOVID.
weird bias by authority aside, do you really think that the daily decisions in this world today, and anytime soon, will be made based on the "let humanity flourish axiom"?
In daily decisions perhaps not, and certainly not when we were in the cave. But from time to time, we will face fundamental questions that need addressing clearly and decisively, and I believe it’s the case with AI. The more advances we make, the more frequent such questions will arise. That is also one of the main reasons why we don’t see any alien civilization. It’s getting harder and harder to keep the entropy low as we gain more knowledges and cognitive power.
How's humanity tackling that climate change question then? Lots of talk. Even lots of action I would argue. And yet, not enough.
Why do you think that's going to change with discussions around AI when there are trillions of dollars at stake?
If Elon Musk was primarily driven by making more money, there'd be many easier ways than what he's been doing and is actively working toward doing
Whether you think he's evil or good or whatever, at least we can agree that he's smart enough to know how to make a lot of money from his existing assets without working 12~18 hours doing one moonshot after another.
> working 12~18 hours doing one moonshot after another.
Does anyone actually beleive Musk works 12 - 18 hour days?
He hires the right people and his companies solve hard problems, but they make a few electric cars, design and build some rockets, and launch some satelites. Most of that shouldn't require much of Musk's time since he has experts working on it.
Other than designing new rockets, there doesn't seem to be many new innovations coming out of Musk-owned companies.
To me, it feels like we all think he must be doing a ton of work even though his companies don't seem to be generating much output.
beyond a point of absolute comfort, the point of money is to accumulate power and power can be spent to make more money. musk doesn't need more money, he needs more power. dangling starlink to ukraine treasonously is that power that money can't buy but space x can.
moonshots are the external story told to maintain social acceptance. tax credits for spacex would be lot harder to sell to taxpayers if everyone unambiguously knew its just another profit seeking private company. tesla gets a lot of lenience because of its supposed pushing of frontiers.
musk can exchange his social capital to manipulate stocks to make others money in exchange for backchannel favors.
idk if musk actually believes mankind will terraform mars but he sure seems to be getting a lot back on earth from saying it.
Nobody in their right mind would start a rocket company in order to make more money. I feel like he just wants to play real life Factorio and he wants everyone to tune in on his Twitch account and watch him play. Meanwhile he was trying to fix democracy [1] (and protect his own companies) by buying Twitter and at some point he realized he fucked up.
But who knows, I could be wrong.
I'm not exactly a fan. I'm not exactly against him either. He just seems like a hardcore geek to me that got way too much money and way too much buy-in.
[1] I'm not saying I that I think he was fixing democracy. I'm saying that I think that he thought he was fixing democracy.
Meanwhile he was trying to fix democracy .
He was literally giving checks for people if they voted a certain way and promoted it.
There is not more anti democratic than this.
you should really listen to one of his full town-hall videos and reflect if anything he said could not have have been said by any of the Clintons or McCain. Then compare the few differences to what you might think of Elon Musk.
This is really a matter of taste, but i (personal opinion) think that Zuck, Karp, Thiel and others who (partly by sucking the inauguration dick) have the ear of the current powers that be, produce much more demented shit than Musk. But the exact personalities really don't matter, the whole jolly crew running the show does not seem very humanistic in their writings or actions.
> how to make a lot of money from his existing assets without working 12~18 hours
This is the biggest lie in today's society. "Make money from assets". There is no such thing. Money cannot work. People can work. He's figured out how to extract a lot of money from his employees, without them realizing the trick that's being played on them.
Effective at improving the planet? Living their own lives better? Or generating profit for him? I'm sure the plantation owners of the american south had effective workers as well...
Everyone involved in AI should have read The Library of Babel [1].
It's about many things, but perhaps the most relevant idea here is that no information matters without understanding. We could generate all possible knowledge, but unless someone--a human--can verify and understand it, it doesn't count. The cure for mortality could be written on the moon, but if no one reads it, it hasn't really been discovered.
The part that does count is that we can verify that we does what we want it to do. We just don't know how.
Same with the black box part of AI. What the weights represent? Arcane dark magic if you ask me. What do they do? Well with LLMs we're all experiencing it.
I find it hard to distinguish between knowing how something works, and being able to predict its behavior (including ways to create and destroy it) with probability approaching 1
> but unless someone--a human--can verify and understand it
Why though a human? Mathematical proofs are very ivory-towery, but if OpenAI would solve a subkind of Cancer, without mortal humans understanding, cancer is still be healed.
Not unless humans were able to produce the medicine or perform the surgery or whatever is necessary to actually heal someone. In order for that to happen it has to be verified and understood by humans, like the other commenter said.
Does it, though? What if future AI can do everything from designing the cure to building a self-contained universal surgical medcapsule? "Just lay down and I'll fix your cancer real good, trust me, bruh." If it was shown to work, would you use it?
Well sure, if it's already proven why wouldn't I use it?
The AI we currently have can't do that though. The AI we currently have is a glorified chatbot. It can't fabricate anything and I doubt it could even reliably design simple real world devices.
It's a really cool and useful technology, and maybe one day it will be as good as you describe, but right now it just isn't and there's no guarantee it ever will be.
I don't think this necessarily follows. The point of the Library of Babel is more one of permutations/combinatorics than of knowledge accumulation itself. In 'our' Library of Babel, each book would be informative, if not perhaps flawed in some ways. That's quite different from one in which in which the number of books that contain anything coherent whatsoever cleanly rounds to zero.
I'm a little more flexible, if the new knowledge (that human's don't understand) can be put into a mechanism and have an observable effect, I'd be happy enough. e.g. a new type of rocket fuel that burns 1000x more efficiently.
If some LLM somewhere managed to produce a coherent and _correct_ explanation of how it worked inside, but no human ever saw it, would it matter?
We know how to apply LLms to problems, which is a subtly different thing to understanding how they do what they do. It's similar to fire: I can cook using fire, but I don't really understand how fire _works_. Heat+oxygen+fuel, sure, but what goes on chemically? I dunno. Doesn't stop me using it. (Pretty sure _humanity_ knows how fire works, though).
If I vibecode a video game and manage to sell it on Steam, the information definitely mattered even though I didn't understand any of it.
There are drugs that nobody truly understands how they work and they are being used by professionals in actual treatments literally right now. We know purely statistical facts like "if drug X is used for condition Y it will help Z% of patients" and can only speculate as to their mechanism of action. They are used anyway and still benefit a lot of people.
The regurgitated approximation can matter just as much depending on what results it accomplishes. If I somehow make money out of this, then it's mission accomplished. Arguably, they might even matter more than my artisanal hand crafted software projects which never paid me a living wage.
But we have pretty good understanding of the logical foundations and the truth of computer generated math. So having a big library which humans could never produce or fully understand has still a value on its own. It is just another level of abstraction.
Some things can simply be beyond humans, but what if an AI could understand it? Just because we don't understand something, it does not mean that it does not matter.
> OBSERVATION: There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species.
Does the set of intelligent species only have one member? I cannot treat this observation as a general rule if there is literally one example of an intelligent species to theorize about.
My problem with this is that yes, looking at the species level from our sample of ~1 the most intelligent species wins out (although there seems to be a lot of debate as to if Neanderthals were more or less intelligent than humans).
But within the sample of the human species it is more common for less intelligent people to control more intelligent people.
This happens within political systems (political leaders are usually above average intelligence, but hardly the most intelligent) and within companies and other economic systems.
I don't see this fairly obvious point discussed, and not really sure what it shows.
it shows that you don't get anything for free. If it was without any cost in other areas to make a human massively more intelligent, it would presumably happen over time via the same processes that led to us.
As it is, the higher (and lower!) ends of the distribution tend to be highly correlated with other issues (mental+physical), and the further you go, the more unfortunate things which make it harder to do stuff in general start to crop up like psychosis, autism, ocd, anxiety, addiction etc
I think the quoted part is saying that given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation. It doesn't suggest "the set of intelligent species only [has] one member".
Yeah but there is only one species that ponders its future.
I stopped reading at this point because I reject this initial assertion, and I assumed everything that follows is based on it.
There is nothing to suggest that an AGI will care about its future or who controls it. (whatever that means). that just more anthropomorphism.
We care a lot about our future so an ai pretending to be like us will also pretend to care. But we have a lot of biology driving us. Our emotions (Fear, Hate, Love, Pride etc) are baked into us in a way just wont be the same for ai.
People keep talking about this like there's a finite amount of math to be done, and then the party's over. But that's never how math has worked, is it? Every problem you solve, ten new ones open up. Like a fractal, the more you zoom in, the more detail emerges. No matter how much better AI is at solving problems, it's not going to generate the "final, complete compendium of mathematics" that that seems to hover over this post.
Math is meaningful because ... some people like to do it. The same as any other human pursuit. It doesn't need a reason beyond that. And AI won't change that. There will continue to be things to explore, things to find out, things that are maybe just at the edge of AI's reach and needs a human to decide whether it's worth continuing to explore or not. (Remember, AI isn't free).
So, IDK, I think for people who enjoy exploring math, there will always be interesting areas to explore. AI just gives us a better flashlight.
BTW I do agree that there's going to be an incident soon, whether intentional, accidental, or paperclip-factory, that leads governments around the world to shut all this down for some time, perhaps even shutting off access to GPUs entirely. It seems unavoidable. But that's just a temporary respite and skirts the core philosophical premise of the post.
> Every problem you solve, ten new ones open up. Like a fractal, the more you zoom in, the more detail emerges. No matter how much better AI is at solving problems, it's not going to generate the "final, complete compendium of mathematics" that that seems to hover over this post.
I guess Terence's main point has been all the time that if we let AI solve all these existing problems, we don't notice the new ones and then there is stagnation.
Because the bounded universe creates bounds on what the intelligences within it can conceive of as mathematics.
We can conceive of lots of mathematics that our own universe doesn't necessarily support. But only that much and no more: we're still made of stuff inside the universe, and so are our tools, so there are upper limits on our conception.
The universe imposes strict limits on the math that can exist within it, and certain broader limits on the math the creatures and well-organized sand within the universe can conceive of in the first place, whether or not it can maybe exist in other universes.
At those levels math and physics are the same bound: the bound of things the universe allows to be conceived of inside it.
It’s possible the math our monkey brains + sand can ever conceive of in this universe is a low and accessible amount.
> The universe imposes strict limits on the math that can exist within it, and certain broader limits on the math the creatures and well-organized sand within the universe can conceive of in the first place, whether or not it can maybe exist in other universes.
When you have a moment, please reference some proofs supporting this.
Trivially false. Let P be the set of maths problems and I be the interesting subset of P. If I is finite, then there exists an element x belonging to P\I whose description is minimal among P\I. Then x is interesting. QED.
I think that's a variation on the interesting numbers paradox joke.
Statement: All numbers are interesting.
Proof: Assume by contradiction that there's a non-empty set of uninteresting numbers. Then that set contains the smallest uninteresting number. That property makes it interesting.
Yeah, I've seen this before as well. I guess I've just become old and grumpy and can't appreciate jokes like these anymore. Also taking jokes seriously is peak HN so..
An interesting problem must have a description that fits in a brain, at least for now. Your description-length argument assumes arbitrarily large storage.
Sorry, I assumed the inductive construction was implied; you can indeed describe properties of that particular interesting problem (though of course you can’t hold its definition in your head), so it goes in the list. Keep going. At some point you’ll hit problems where the process of constructing the problem doesn’t even fit in a brain, etc. There are at least countably many problems, but finitely many problems which any algorithm-which-fits-in-the-brain can describe given finitely many inputs-which-fit-in-the-brain.
This isn’t an enormously important point - the actual question at issue is an empirical one, “in a steady state, can we produce interesting problems at a rate that exceeds our ability to solve them and integrate our understanding” or something like that - but I did rankle at a “trivial” proof which is invalid due to equivocating between multiple definitions of the word “interesting” (which should really take an object, “interesting to me” vs “interesting to something smarter than me”).
I think it really depends on what the universe looks like as you drill down into it. It seems like the further down into smaller systems you get, the more analytically complex it gets. And then there will always be more value in enhancing the generalisations you have.
I would argue that novel and/or valuable results are not necessarily interesting!
I (a human) am interested in things that are applicable to my realm of understanding, but I see a very plausible future where novel and/or valuable results leave that realm.
I'd further argue that's already the case for most math for most humans. What's interesting to Terrance Tao is rarely of immediate interesting to me.
Consider a world with one person in it. That person defines the jobs to be done. As soon as that person is disabled for one reason or another and is dying due to lack of care, there are by definition more jobs to be done than people to do them. "Businesses" and "institutes" are just constructs that exist to summarize those needs at scale.
I find Tao's writing on this very helpful, particularly because a lot of the ideas generalise to most knowledge work. It is darkly fortunate that some of the smartest and most articulate people on the planet are affected by the AI race - it gives me a little hope that alignment and control can be solved.
Companies are nothing other than the tool of rich people used to control normal people. There's some sort of mass delusion that "companies" are evil. Guess what, a company has no own agency to destroy the environment, exploit people, profit from suffering, etc. It's just following the whims of its shareholders, who are humans with accountability.
AI does NOT “brute-force”, else we could have had those math breakthroughs decades ago. Seeing this dishonest goalpost-moving every few months is so tiring.
> Similar logic applies to every industry and every job. And it comes to the conclusion that we won’t have enough people for all the jobs that need to be done.
In the (extremely) short run, yes. In the long run, those jobs will also be done by AI.
It's like chimpanzees seeing human society and saying "look how complex it is, imagine how many chimpanzees it needs to maintain it".
> In the (extremely) short run, yes. In the long run, those jobs will also be done by AI.
Let's be honest: we don't know. Maybe you're right, but for the moment it's more likely that you're not. And countries cannot bet on that vague intuition at the cost of destroying their research communities and world leadership (which takes decades if not a century to achieve).
Math proofs are special because they’re verifiable, safe, and do not require physical experiments. You can perform exhaustive parallel search in simulation with RLVR.
Do you think this applies to say, surgery, as well? There are few useful problems that share these properties.
First off, I can’t imagine anything more torment nexus-y than throwing billions to automate and scale the torture of animals. If each token is a “cut”, how much suffering does 10 trillion training tokens (lower bound) corresponds to?
Second, this still doesn’t cover all the properties that make math proofs doable. It requires working in the physical world. You must physically capture or grow 10T cuts worth of animals. You cannot verify success so easily, either. Cancer cells, for example, could regrow over months. You would need to keep the animal alive and regularly test the animal, which would be difficult to scale. And most people wouldn’t trust the world’s greatest vet to operate on them, anyways.
I'm not advocating for it, I don't know enough. But we already slaughter lots of animals and raise them in horrible conditions, and all robot assisted surgery already goes through animal trials.
> Second, this still doesn’t cover all the properties that make math proofs doable. It requires working in the physical world. You must physically capture or grow 10T cuts worth of animals.
It would likely be sim2real with that as the post training, reducing that requirement a lot.
> Cancer cells, for example, could regrow over months.
In that specific scenario you would likely train on receiving no unrelated injuries during the surgery, and have induced conditions with stuff tagged molecularly that you can then verify efficacy from without waiting months.
Depending on how much more data efficient sim2real makes it, you could end up seeing companies pushing it only for actual procedures the animals need but economically would never get; botched surgery and the animal gets euthanized before waking up, which they could argue was already going to happen.
> And most people wouldn’t trust the world’s greatest vet to operate on them, anyways.
Robotic surgery systems, already go through animal trials before being used on humans. So do many purely human surgery techniques.
A few people with way to much power are building AI to achieve their utopian, sci-fi dream. A few other powerful people are using that first group to achieve their post-democratic goals. It’s the most obvious top down attempt at imposing their vision onto the rest of the world. They aren’t building tools to serve anyone else than themselves
No, I mean utopia. For them. A dystopia is what we will all experience, when the utopia they are pitching fails to materialize. I’m using the traditional, correct definition of the term, where a dystopia is a utopia that failed to achieve a paradise for the people and instead becomes hell
There's one flaw in the evidence for the logic chain. The hugging face attack is used to demonstrate three things: the need for oversight today (fundamental to the article) and to demonstrate some kind of drift or unexpected capability gain, and finally to hint and some fundamental morality of the AI or at least the risk of drift from our morality.
I'd argue that what the hugging face attack illustrates is that large AI companies are motivated to have bombastic claims supported by bombastic demos. The model was clearly trained or encouraged to work as it did, as evidenced by the fact it keeps using this particular escape hatch.
And the fact that it aligns with prior and current calls for what very likely might be a regulatory capture / oversight capture move right before IPO. It aligns so well with this "barely constrained superweapon" narrative it might as well be PR.
How does your logic explain that OpenAI seems to want to hide the extent of the HuggingFace incident, and every week we learn from 3rd parties about new victims of the hack?
4D chess? They want others to find the hacked services, so the report of how dangerous the agents are seems more "legit"?
IMHO hiding details of the hack helps conceal that they built it to do what it did, and maybe were able to know it was working just as intended.
The fact it keeps doing it, with more and more evidence, is a sign that it's built that way.
This is a program running on their montoroed machines that they purpose built and monitored its training at every step. I think it'd be way more suprising that they didn't know it used note taking and cross-run memory.
AI is not AGI right now, it can't think or come up with something new like a human brain. It still uses knowledge that was made by a human and was published on the Internet.
You're confusing the mechanical language of math with math itself.
The map is not the territory, etc.
If math was just an elaborate linguistic Glass Bead Game then we wouldn't be funding it. The intuition is that the surface rules of math help uncover the underlying structure of reality.
Because it’s high time to rethink the incentives and rewards structures to nurture the next generations of mathematicians when corporate money and automated AI systems are mining the pool of limited interesting problems without giving back or enhance our understanding.
The problem they care about is not just “AI replaces human mathematicians” but rather the society thinks mathematicians can be replaced by AI.
I think this axiom is of course true. But the mistake the article makes, in my opinion, is to try to apply this axiom separately to each domain. If we have this as the over-arching axiom, it is not clear at all that humans should be steering the development of mathematics. Maybe it would be better for humanity if the department of world math is run by AI.
You're talking about this article? I don't see how you could get that, fundamentally I read it the other way around. Unless you're talking about the base assumption of capabilities.
HN is now heavily astroturfed or overrun by bots (or, alternately, low-quality posters now indistinguishable from the previous), and this is especially so in AI-related threads.
One tell is that most comments barely exceed one or two sentences (because otherwise AI detection gets easier and much more reliable), when this was not as much the case many years ago. The drive-by comments are also low / zero quality, mostly expressing a feeling or agreement/disagreement, and primarily driven by ideology or pre-existing beliefs and commitments.
Look at non-AI-related threads and you'll notice a large distribution shift relative to AI-related ones.
EDIT: Basically HN is orange Plebbit now. If you doubt this, compare HN discussions to those on e.g. lobste.rs, LessWrong, The Motte, DSL, ACX, or other old obscure forums. You'll notice those places have their own very serious biases and problems, but it is obvious the vast majority of posters are nevertheless human and making some minimal efforts.
Now compare Reddit and 2026 HN to the above, and see if you can confidently say the same.
Friendly warning to those who might not be aware: the vast majority of comments below posts like the above will be left by (otherwise intelligent) programmers who think mathematics is a closed system where one attempts to solve endless Olympiad-type problems. I wouldn’t take any of it seriously at all. Better to listen to what those who actually know what the subject is about have to say.
Unfortunately, mathematics (especially pure mathematics) is by its very nature very, very poorly understood by those who haven’t worked as a mathematician. Even worse, those who don’t understand are seemingly not at all aware of their misunderstanding and are entirely confident in their (very wrong) characterisation of the subject.
I did systems administration for a university math department for several years. I came to the conclusion that mathematics (and perhaps philosophy) were both topics where it was likely that no staff members in that department could describe "what goes on here" and that possibly even within the department, one professor may not be able to describe what another professor's actually doing.
The closest I could come to describing math is "some abstract process where imagined structures are characterized and extended; the most critical part of the process is identifying where seemingly independent structures are found to actually be fungible in some previously undiscovered way".
A simple example is
"hey, did you know that x^i is the unit circle?"
"what's i?"
"i is defined as if you square it the result is -1"
"what does that have to do with circles?"
> Mathematicians can also consider wholly redirecting their skill sets to work on real world problems. I’ve actually been encouraging mathematicians to consider thinking about working on government or other large-scale societal issues.
The fact that this is a radical departure from the norm is part of why mathematics (and philosophy) is often seen as some intangible or ungrokable science to many outsiders, as they're generally approaching it from a perspective of "Okay, but why, what is this useful for?", and the answer "For the science of it" doesn't tend to land with people that aren't already passionate about said science/discipline and are just trying to figure out what it even is or involves.
Doesn't help that there is a pervasive sentiment in American Academia (not sure about elsewhere) about Math being *the* hard science, and I mean hard as in difficulty, so a lot of people get intimidated by it before they ever give it a chance very early on in their academic life and carry that through the rest of their education.
So there ends up being a rather small pool of people that are in(to) the field, and rather high friction for stimualting interest in it from outsiders from the way that it's taught, and a massive difference in the perspective of it's use between it's diaspora and the unmathed masses.
Mathematicians who do work on "real world" problems are mostly doing so with theoretical physicists, genomicists, and cryptographers. None of these count as what people consider valuable other than because they are hard.
That said, few 50 (or even 40) years ago would have predicted that completely abstract number theoretical computations about primes, discrete logarithms, and elliptic curves would be the foundation of our monetary system.
> I did systems administration for a university math department for several years. I came to the conclusion that mathematics (and perhaps philosophy) were both topics where it was likely that no staff members in that department could describe "what goes on here" and that possibly even within the department, one professor may not be able to describe what another professor's actually doing.
And this is indeed why it is not going to be taken seriously as an academic or (more importantly) an economic endeavour done by humans anymore.
That won't stop the career mathematicians from protesting and having a cry here trying to justify themselves.
Good question, and maybe I should have provided an alternative description rather than just criticising.
Unfortunately, it is actually surprisingly hard to pin down, and I think mathematicians (and, as a student, I count myself as one to some degree at least) now have the task of making this a lot clearer. If we want to justify our existence in the face of new machines that can seemingly ‘do our work for us’ (so far in a restricted context), we should give a robust defence of our practice. If we can’t do this, we simply don’t deserve the funding (which, by the way, again contrary to some misguided statements here, isn’t very much anyway!). I think all of this will become clearer to outsiders as time passes, but for now it’s not easy to give a quick answer — though I can try.
Mathematics is about understanding things. Isn’t that what every subject is about? Well, I suppose so, but mathematics more specifically does something like the following:
(1) observe some phenomenon in ‘reality’.
(2) attempt to formalise that phenomenon in such a way that it can be manipulated purely symbolically.
(3) use this (perhaps fairly arbitrary; remember that we can invent as many formal systems as we like) system to deduce from our initial assumptions new facts that would otherwise have been very non-obvious.
It seems like outsiders have a decent grasp of (3) and the application of AI to it, but have very little idea about the other two steps. It seems to be widely assumed among non-mathematicians that problems are essentially god given and that the job of a mathematician is therefore to chug away on these problems, manipulating symbols and trying out tools, in the hope of learning a yes/no answer to each one.
The first two steps are by far the hardest and most important, and they’re also the parts that AI seems currently unable to help with.
NOTE: this is not a deeply insightful description of what the subject is about, and there are many better characterisations out there. I think Tao and various others have written recently about why complicated and inscrutable AI-generated proofs aren’t nearly as valuable as one might imagine. (That’s not to say there’s no value to such proofs; perhaps in time, as technology improves, mathematicians will come to accept AI as part of the process.)
If you want to understand all of this issues better, reading the recent slew of guest posts on Tao’s blog would be a very good start.
Despite loving Mathematics and having considered being a Mathematician myself, that wasn't a very robust defense. Also, even though I'm a big Terence Tao fan, I'm not sure he has sorted that out this defense in full himself.
This post which he forwarded was quite poor in my opinion. Confusing, all over the place with AI criticisms and promotion of the AI hazing being done by mathematicians.
X thousand mathematicians who want to protect their livelihoods signed a bunch of letters against AI. Duh. We've seen similar movements from every profession that has been displaced ever.
Terence Tao uses AI and has made a few good points on how to use it. But defensiveness leaks into almost every defense of the role of humans in Mathematics that I've read, even his own at times.
To be clear, I actually believe that Mathematicians aren't going away, but I dont have enough knowledge about the life of a professional mathematician to articulate a path forward.
This "path forward" is what I'd like to see. We need a top mathematician with enough intellectual honesty (Terence Tao qualifies, I think) to start this questioning with "there's actually no role for human Mathematicians" as one of the options on the table and go from there.
For me, mathematicians seem to be still having an inner discussion rather a making these essays for the more general public. In any case, I think that statements of the type "there's actually no role for human Mathematicians" are completely non-serious, so it'd sad that the discussion concentrates on that.
I agree, but it's still something we shouldn't eliminate a priori. I think Mathematics as a profession will be greatly enhanced by AI, but I can't prove that.
My understanding when I was practicing is that the trend in modern mathematics is to focus on spaces with a certain kind of structure, and maps between those spaces that preserve it, and then what are invariants are preserved by those maps. Structure-preserving maps between categories of such spaces - "functors" in the language of category theory - are especially neat.
That's certainly different from Olympiad-style problems.
Actually, if you were to convert most software into mathematical notation it would look incomprehensible. We are not interested in gatekeeping as much by making things look more convoluted than they need to.
This is a great point. When I was learning machine learning in the early 00s, there were many papers where I would struggle to understand exactly what the mathematics was trying to communicate, but where I would look at the matlab source code and say to myself "that's all?"
In my observation, most programmers and engineers lose their math chops over time. The math they need is mostly baked into their tools, such as CAD. If a problem requires more advanced math, it's given to a "math person" in the department. Often, the "math person" is also not allowed to touch the production code.
i'd be on board with this concept but the author seems to believe his point is generalisable to all under industries, hence committing the same fallacy you're talking about at a large scale.
but you're still right. i disregarded his take, as you would with mine re. math.
I am shocked how people can deny that solving Navier Stokes requires some sort of intelligence. Even Doctorow talks of "brute-forcing" a solution. Brute-forcing leads to combinatorial explosion, so there must be something more going on here. Otherwise you could just put this problem into an automated theorem prover (we've had those forever, they are actually just brute-forcing it).
> Otherwise you could just put this problem into an automated theorem prover (we've had those forever, they are actually just brute-forcing it).
There are many automatic theorem provers that do very clever stuff, just as the underlying theroy describes.
> I am shocked how people can deny that solving Navier Stokes requires some sort of intelligence.
It is absurd to waste time discussing whether it is inteligent or not. It is just an algorithm, we know how it works, and it does exactly what we expect it to do. LLMs are not magical things. The main difference is the scale: for Navier-Stokes they spent in 3 days more money that the whole mathematical community over the last 20 years easily.
By the way, I'm not saying that LLM's are useless, that I'm anti-AI or anything like that.
Intelligence does not seem to be magic either, as LLMs are proving now. It is indeed a waste of time to argue that LLMs are not intelligent in their own way, they obviously are. If Navier Stokes doesn't convince you, nothing will.
I just used a £89 Codex subscription to do very intelligent things with it, stuff that I would have had to sit down and ponder and work on for quite a while, and I have a PhD in that. I didn't need to do anything special except explaining the problem(s) to the AI, and my theory of it so far. It took it from there. If that is not intelligence, nothing is.
'There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders control ... to the less capable species'.
True with genuine species. But we should take note of the plentiful counter examples within human societies. How about politicians and our method of choosing those people to whom we delegate the most critical decisions regarding our and the Earth's future? We select politicians mostly either by rote or via their persuasive rhetoric, their general personality & likeability and probably least of all by their intellectual capability or indeed general capability in too many cases. That is not to say intellectuals are necessarily any better at the job. There are numerous other examples in human organizations as we know, sometimes to our cost. Truth is that we cannot even agree on how, as a species together with the other life forms on Earth, we can all 'flourish' though there are lots of great examples working in local environments.
I sometimes think about this: to grasp the fundamentals of knowledge and complex phenomena, perhaps we need an external system rather than human knowledge systems. By that logic, maybe we need AI, which can handle far greater complexity.
Human capability, when you think about it, is complex. Why is Newton praised as being so damn great? He established the law of universal gravitation, F=ma. Why is that such a big deal?
He distilled countless phenomena in an open system into a single mathematical formula.
What makes it great is that he found common state variables and relationships across entirely different phenomena like falling objects, planetary motion, collisions, and artillery trajectories.
But does F=ma hold true for the entire macroscopic world? No. There are various conditions and specific situations in motion, but within most scenarios and a certain range of approximation, it outputs values that are useful to humans.
Why is the Schrödinger equation so great? Because it turned the time evolution of quantum states into a calculable mathematical law.
Human thought is essentially creating a closed system by deciding what to cut out and what to keep from the infinite degrees of freedom in reality. Academia is what reinforces that closed system.
A great theory is great not because it perfectly replicates reality, but because it compresses the immense complexity of reality into a small, closed formal system while still managing to explain a multitude of phenomena.
In that process, it feels like human thought and progress are shifting into a different framework.
What LLMs do well is primarily exploring within the ontology and representation space that humans have already built.
I think there are two broad categories of discovery:
One is forming a new closed system, and the other is connecting fragmented knowledge within that closed system. I feel that the vast majority of research focuses on the latter.
What LLMs excel at is finding unvisited points within a given representation space. This is typically the process through which master's and PhD students connect dots, build their skills, and form their own mental models. But the logic behind criticizing LLMs seems to be that they eliminate the very work these graduate students need to do in order to grow.
However, looking at it from another angle, perhaps our current knowledge systems and classifications have reached a limit, suggesting that we might actually need a completely new classification and knowledge system.
What is the core principle of an LLM? It's predicting the probability of the next sequence.
Let's say you type the word "cat". Cat - is cute (90%), want to eat it (6%), furry (4%). Because "is cute" has the highest probability, the next sequence proceeds in that direction.
Within this framework, human knowledge and logic largely operate the same way. Once an initial logical proposition is established, we follow it up with whatever makes logical sense next. From that perspective, I think LLMs will actually do this better.
But what is it that LLMs cannot do right now? They cannot create that initial logical proposition. I believe they lack the ability to carve out a closed system from an open system.
Stacking logic step-by-step within a closed system—LLMs do this exceptionally well. But whether that constitutes true "intelligence" is a different matter.
I feel that being logical does not necessarily equate to having intelligence.
Humans preserve and create different mental models and knowledge systems within an open system. Just as your thoughts differ from mine, LLMs lack the ability to form these distinct mental models.
If so, within these limits, what humans must ultimately do is construct the logical frameworks that LLMs can then fill in. Perhaps a new kind of logic dedicated to designing these frameworks will become the next major trend.
Viewed from this perspective, I have no idea if we are in a mere technological transition or something else entirely. Or whether it is even correct to say humans are strictly necessary to build that framework. Maybe my learning is just lacking.
Why do we need any job? Check "Bullshit Jobs" by David Graeber. Not saying being a mathematician is a bullshit-kinda job, but there have been so many made up positions way before this AI-era. In that sense AI isn't changing much.
Most people work for a living. Then there are people who find meaning in work. And people who advance in life by working. You want to do away with all that?
Well it's actually nice, maybe more people will be able to do world class math with the help of these tools. There are few fields as elitist and hostile as pure mathematics, most mathematicians I know build their whole life around their profession and their self esteem is strongly coupled to the fact that they can do things that most other people can't. Naturally, many will be devastated when (if) you take that away from them. That said I think AI is still overhyped and human mathematicians can easily outthink it in most domains, look at how difficult it is for an AI to write even a single decent paper, a good PhD student can easily outclass it in that regard. All of these impressive results were generated by having world-class mathematicians steer the systems using highly tuned prompts, so I see it more like a super violin that produces beautiful music when played by master violinists rather than being a fully autonomous orchestra which many people are led to believe this already is.
And maybe let's not only hear the opinion of two or three Fields level mathematicians with blogs, 99 % of the worlds mathematicians in academia might profit from these tools as they might partially close the gap between them and the world elite, making creativity and tenaciousness more important than having the right neocortical structure allowing you to outperform 99.9 % of other humans at keeping context in your head and making predictions, AI can do that better now with the right prompts.
We need to pivot math towards intuition and accessibility.
Math has built a gated, inaccessible institution which - by design or not - served as a moat.
I believe math has little to do with mathematical notation - intuition is much more important. One can have intuition but not be able to “read math” - much as many musicians don’t “read music”.
Reading math however does not guarantee ideas or intuition. And those are what math needs.
I think LLMs are the single best things that were invented to make mathematics accessible.
I loved mathematics when I was young and wanted to be good at it, but I got burnt so many times by teachers who could not explain even if their life depended on it.
Asking an LLM and discovering how different theories belong together and what are genuinely open questions, along with different philosophical interpretations has been a wonderful gift.
I have long accepted that Mathematics is a language where you must go "all in" because for the most parts institutions are fundamentally unable to explain it in an intuitive way. There are people who "just get it" and are blind to the struggle of people who need a different approach in learning it. There's something about the field where a generational survivorship bias has created an environment where math is mostly taught to likeminded people, who then don't see why anybody could not see the difficulties learning it.
I know this is not true everywhere, but the only reason I could finish my studies was because there are some genuinely excellent online lectures on youtube, along with massive support from friends who have already been through the ordeal.
If I had had access to an LLM, I could have actually taught myself from scratch and be more playful in learning it. And no, I am not willing to learn lots of math "for fun", I can think of better "for fun" things that actually help my profession and cater to my actual interests and not just some academic ideal.
You would be surprised at the number of people (who study math formally) who just memorize proofs and having no clue what it all fundamentally is about.
> We (humans) should help humanity flourish. I understand that not everyone agrees. I have been called a “speciesist” for being “too human-centric”.
this is insane. the "speciesist" people aside (the best way for all other species to flourish is for humans to eradicate themselves right now, which is completely mad), i have the opposite problem: the _axiom_ seems to be: we should help ME flourish. "Me" as in people who are raking in trillions for their own very special selves right now, at the cost of everyone else's future, while none of them can be trusted to hold my cell phone for a second.
> if an industry commits to the axiom of helping humanity flourish
where does he see such industries, outside of maybe nonprofits?
> the best way for all other species to flourish is for humans to eradicate themselves right now
That wouldn't do much for rice, pigs, chickens, or rats.
As I understand you referred to Elon Musk. But this is Poh Shen Loh talking. If you don’t know who he is, look up or at least read his articles.
From his Wikipedia page:
> Po-Shen Loh (Chinese: 罗博深; pinyin: Luó Bóshēn; born June 18, 1982) is an American mathematician specializing in combinatorics. Loh teaches at Carnegie Mellon University, and from 2014 to 2023 served as the national coach of the United States' International Mathematical Olympiad team. He is the founder of educational websites Expii and Live, and lead developer of contact-tracing app NOVID.
weird bias by authority aside, do you really think that the daily decisions in this world today, and anytime soon, will be made based on the "let humanity flourish axiom"?
In daily decisions perhaps not, and certainly not when we were in the cave. But from time to time, we will face fundamental questions that need addressing clearly and decisively, and I believe it’s the case with AI. The more advances we make, the more frequent such questions will arise. That is also one of the main reasons why we don’t see any alien civilization. It’s getting harder and harder to keep the entropy low as we gain more knowledges and cognitive power.
How's humanity tackling that climate change question then? Lots of talk. Even lots of action I would argue. And yet, not enough. Why do you think that's going to change with discussions around AI when there are trillions of dollars at stake?
If Elon Musk was primarily driven by making more money, there'd be many easier ways than what he's been doing and is actively working toward doing
Whether you think he's evil or good or whatever, at least we can agree that he's smart enough to know how to make a lot of money from his existing assets without working 12~18 hours doing one moonshot after another.
> working 12~18 hours doing one moonshot after another.
Does anyone actually beleive Musk works 12 - 18 hour days?
He hires the right people and his companies solve hard problems, but they make a few electric cars, design and build some rockets, and launch some satelites. Most of that shouldn't require much of Musk's time since he has experts working on it.
Other than designing new rockets, there doesn't seem to be many new innovations coming out of Musk-owned companies.
To me, it feels like we all think he must be doing a ton of work even though his companies don't seem to be generating much output.
beyond a point of absolute comfort, the point of money is to accumulate power and power can be spent to make more money. musk doesn't need more money, he needs more power. dangling starlink to ukraine treasonously is that power that money can't buy but space x can.
moonshots are the external story told to maintain social acceptance. tax credits for spacex would be lot harder to sell to taxpayers if everyone unambiguously knew its just another profit seeking private company. tesla gets a lot of lenience because of its supposed pushing of frontiers.
musk can exchange his social capital to manipulate stocks to make others money in exchange for backchannel favors.
idk if musk actually believes mankind will terraform mars but he sure seems to be getting a lot back on earth from saying it.
Nobody in their right mind would start a rocket company in order to make more money. I feel like he just wants to play real life Factorio and he wants everyone to tune in on his Twitch account and watch him play. Meanwhile he was trying to fix democracy [1] (and protect his own companies) by buying Twitter and at some point he realized he fucked up.
But who knows, I could be wrong.
I'm not exactly a fan. I'm not exactly against him either. He just seems like a hardcore geek to me that got way too much money and way too much buy-in.
[1] I'm not saying I that I think he was fixing democracy. I'm saying that I think that he thought he was fixing democracy.
Meanwhile he was trying to fix democracy . He was literally giving checks for people if they voted a certain way and promoted it. There is not more anti democratic than this.
right because true democracy is when you give cheques to people to convince other people to vote a certain way
Well, he is famous for axing down middle management in companies. Might as well get rid of Super PACs and buy voters directly.
I don't understand how those two are connected. Both are bad though.
you should really listen to one of his full town-hall videos and reflect if anything he said could not have have been said by any of the Clintons or McCain. Then compare the few differences to what you might think of Elon Musk.
Yet he has got richer on the back of spacex …
I don’t think we need to over complicate a megalomaniac desire for more power and wealth.
This is really a matter of taste, but i (personal opinion) think that Zuck, Karp, Thiel and others who (partly by sucking the inauguration dick) have the ear of the current powers that be, produce much more demented shit than Musk. But the exact personalities really don't matter, the whole jolly crew running the show does not seem very humanistic in their writings or actions.
> how to make a lot of money from his existing assets without working 12~18 hours
This is the biggest lie in today's society. "Make money from assets". There is no such thing. Money cannot work. People can work. He's figured out how to extract a lot of money from his employees, without them realizing the trick that's being played on them.
Phrased differently, he figured out to make his workers much more effective. You know, doing exactly the job management has?
Effective at improving the planet? Living their own lives better? Or generating profit for him? I'm sure the plantation owners of the american south had effective workers as well...
Effective at doing the jobs they were hired to do: produce and operate vastly improved launch vehicles (in the case of SpaceX).
None of those other goals are in the remit of a company in a market economy. You might as well be complaining that a horse can't sing.
Everyone involved in AI should have read The Library of Babel [1].
It's about many things, but perhaps the most relevant idea here is that no information matters without understanding. We could generate all possible knowledge, but unless someone--a human--can verify and understand it, it doesn't count. The cure for mortality could be written on the moon, but if no one reads it, it hasn't really been discovered.
[1] https://maskofreason.wordpress.com/wp-content/uploads/2011/0...
> unless someone--a human--can verify and understand it, it doesn't count.
One counter-point: Humans have not figured out how general anesthesia works, but we use it every day to great effect.
https://en.wikipedia.org/wiki/Theories_of_general_anaestheti...
The part that does count is that we can verify that we does what we want it to do. We just don't know how.
Same with the black box part of AI. What the weights represent? Arcane dark magic if you ask me. What do they do? Well with LLMs we're all experiencing it.
> The part that does count is that we can verify that we does what we want it to do. We just don't know how.
How do we know the person getting anesthesia doesn't die and a new soul/consciousness replaces them?
(We can ask the same question about going to sleep, er even walking through a door, but it's still something to think about).
you can verify it does what you want it to? How? Everything is a probability and so is your verification.
I find it hard to distinguish between knowing how something works, and being able to predict its behavior (including ways to create and destroy it) with probability approaching 1
Only because we understand everything _except_ how it works.
how can we understand absolute everything and still don't know how it work ?
Trial and error :)
I wouldn't say that is "understanding". We must fully be able to explain to say that we understand it.
There are two types of understanding: knowing how a candle works and knowing how a candle works.
Seems a bit nitpicky, we certainly understand how to use it with great effect, we just don't understand the biochemical mechanism.
> but unless someone--a human--can verify and understand it
Why though a human? Mathematical proofs are very ivory-towery, but if OpenAI would solve a subkind of Cancer, without mortal humans understanding, cancer is still be healed.
Not unless humans were able to produce the medicine or perform the surgery or whatever is necessary to actually heal someone. In order for that to happen it has to be verified and understood by humans, like the other commenter said.
Does it, though? What if future AI can do everything from designing the cure to building a self-contained universal surgical medcapsule? "Just lay down and I'll fix your cancer real good, trust me, bruh." If it was shown to work, would you use it?
Well sure, if it's already proven why wouldn't I use it?
The AI we currently have can't do that though. The AI we currently have is a glorified chatbot. It can't fabricate anything and I doubt it could even reliably design simple real world devices.
It's a really cool and useful technology, and maybe one day it will be as good as you describe, but right now it just isn't and there's no guarantee it ever will be.
I don't think this necessarily follows. The point of the Library of Babel is more one of permutations/combinatorics than of knowledge accumulation itself. In 'our' Library of Babel, each book would be informative, if not perhaps flawed in some ways. That's quite different from one in which in which the number of books that contain anything coherent whatsoever cleanly rounds to zero.
While I get the point and it's true.
I'm a little more flexible, if the new knowledge (that human's don't understand) can be put into a mechanism and have an observable effect, I'd be happy enough. e.g. a new type of rocket fuel that burns 1000x more efficiently.
We already don't understand LLMs and they are having oberservable effects
If some LLM somewhere managed to produce a coherent and _correct_ explanation of how it worked inside, but no human ever saw it, would it matter?
We know how to apply LLms to problems, which is a subtly different thing to understanding how they do what they do. It's similar to fire: I can cook using fire, but I don't really understand how fire _works_. Heat+oxygen+fuel, sure, but what goes on chemically? I dunno. Doesn't stop me using it. (Pretty sure _humanity_ knows how fire works, though).
Doubt.
If I vibecode a video game and manage to sell it on Steam, the information definitely mattered even though I didn't understand any of it.
There are drugs that nobody truly understands how they work and they are being used by professionals in actual treatments literally right now. We know purely statistical facts like "if drug X is used for condition Y it will help Z% of patients" and can only speculate as to their mechanism of action. They are used anyway and still benefit a lot of people.
Yeah that information mattered when someone wrote it down, not when the LLM regurgitated an approximation.
The regurgitated approximation can matter just as much depending on what results it accomplishes. If I somehow make money out of this, then it's mission accomplished. Arguably, they might even matter more than my artisanal hand crafted software projects which never paid me a living wage.
But we have pretty good understanding of the logical foundations and the truth of computer generated math. So having a big library which humans could never produce or fully understand has still a value on its own. It is just another level of abstraction.
Ironies of automation might be better suited to explain the problem.
Update:
"I own nothing, have no privacy, [never have to think, and am not required to solve any problems,] and life has never been better."
https://en.wikipedia.org/wiki/You'll_own_nothing_and_be_happ...
> but unless someone--a human--can verify and understand it, it doesn't count
Does it need to be a human or can it be some other form of life?
It can be other form of life, but that other form of life need to be extremely aligned and keep humanity interest as top priority.
Some things can simply be beyond humans, but what if an AI could understand it? Just because we don't understand something, it does not mean that it does not matter.
> OBSERVATION: There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species.
Does the set of intelligent species only have one member? I cannot treat this observation as a general rule if there is literally one example of an intelligent species to theorize about.
My problem with this is that yes, looking at the species level from our sample of ~1 the most intelligent species wins out (although there seems to be a lot of debate as to if Neanderthals were more or less intelligent than humans).
But within the sample of the human species it is more common for less intelligent people to control more intelligent people.
This happens within political systems (political leaders are usually above average intelligence, but hardly the most intelligent) and within companies and other economic systems.
I don't see this fairly obvious point discussed, and not really sure what it shows.
> not really sure what it shows
it shows that you don't get anything for free. If it was without any cost in other areas to make a human massively more intelligent, it would presumably happen over time via the same processes that led to us.
As it is, the higher (and lower!) ends of the distribution tend to be highly correlated with other issues (mental+physical), and the further you go, the more unfortunate things which make it harder to do stuff in general start to crop up like psychosis, autism, ocd, anxiety, addiction etc
I think the quoted part is saying that given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation. It doesn't suggest "the set of intelligent species only [has] one member".
Yeah but there is only one species that ponders its future.
I stopped reading at this point because I reject this initial assertion, and I assumed everything that follows is based on it.
There is nothing to suggest that an AGI will care about its future or who controls it. (whatever that means). that just more anthropomorphism.
We care a lot about our future so an ai pretending to be like us will also pretend to care. But we have a lot of biology driving us. Our emotions (Fear, Hate, Love, Pride etc) are baked into us in a way just wont be the same for ai.
People keep talking about this like there's a finite amount of math to be done, and then the party's over. But that's never how math has worked, is it? Every problem you solve, ten new ones open up. Like a fractal, the more you zoom in, the more detail emerges. No matter how much better AI is at solving problems, it's not going to generate the "final, complete compendium of mathematics" that that seems to hover over this post.
Math is meaningful because ... some people like to do it. The same as any other human pursuit. It doesn't need a reason beyond that. And AI won't change that. There will continue to be things to explore, things to find out, things that are maybe just at the edge of AI's reach and needs a human to decide whether it's worth continuing to explore or not. (Remember, AI isn't free).
So, IDK, I think for people who enjoy exploring math, there will always be interesting areas to explore. AI just gives us a better flashlight.
BTW I do agree that there's going to be an incident soon, whether intentional, accidental, or paperclip-factory, that leads governments around the world to shut all this down for some time, perhaps even shutting off access to GPUs entirely. It seems unavoidable. But that's just a temporary respite and skirts the core philosophical premise of the post.
> Every problem you solve, ten new ones open up. Like a fractal, the more you zoom in, the more detail emerges. No matter how much better AI is at solving problems, it's not going to generate the "final, complete compendium of mathematics" that that seems to hover over this post.
I guess Terence's main point has been all the time that if we let AI solve all these existing problems, we don't notice the new ones and then there is stagnation.
There probably is a finite amount of math to be done
The universe is bounded by rules, as far as we can tell, and not a lot of them
Why does the universe being bounded imply mathematics is finite?
Because the bounded universe creates bounds on what the intelligences within it can conceive of as mathematics.
We can conceive of lots of mathematics that our own universe doesn't necessarily support. But only that much and no more: we're still made of stuff inside the universe, and so are our tools, so there are upper limits on our conception.
>The universe is bounded by rules
That's physics. Not all math is physics.
Which math is not physics (and chemistry)? Fundamentally, all these are same
The universe imposes strict limits on the math that can exist within it, and certain broader limits on the math the creatures and well-organized sand within the universe can conceive of in the first place, whether or not it can maybe exist in other universes.
At those levels math and physics are the same bound: the bound of things the universe allows to be conceived of inside it.
It’s possible the math our monkey brains + sand can ever conceive of in this universe is a low and accessible amount.
> The universe imposes strict limits on the math that can exist within it, and certain broader limits on the math the creatures and well-organized sand within the universe can conceive of in the first place, whether or not it can maybe exist in other universes.
When you have a moment, please reference some proofs supporting this.
It seems likely that there are an infinite number of math problems but only a finite number of interesting ones.
Trivially false. Let P be the set of maths problems and I be the interesting subset of P. If I is finite, then there exists an element x belonging to P\I whose description is minimal among P\I. Then x is interesting. QED.
Why is x interesting? Just because it has a minimal description in P\I? That makes it interesting in strictly technical sense only.
I think that's a variation on the interesting numbers paradox joke. Statement: All numbers are interesting. Proof: Assume by contradiction that there's a non-empty set of uninteresting numbers. Then that set contains the smallest uninteresting number. That property makes it interesting.
Yeah, I've seen this before as well. I guess I've just become old and grumpy and can't appreciate jokes like these anymore. Also taking jokes seriously is peak HN so..
An interesting problem must have a description that fits in a brain, at least for now. Your description-length argument assumes arbitrarily large storage.
the smallest problem that cannot fit in a brain would be pretty interesting
Sorry, I assumed the inductive construction was implied; you can indeed describe properties of that particular interesting problem (though of course you can’t hold its definition in your head), so it goes in the list. Keep going. At some point you’ll hit problems where the process of constructing the problem doesn’t even fit in a brain, etc. There are at least countably many problems, but finitely many problems which any algorithm-which-fits-in-the-brain can describe given finitely many inputs-which-fit-in-the-brain.
This isn’t an enormously important point - the actual question at issue is an empirical one, “in a steady state, can we produce interesting problems at a rate that exceeds our ability to solve them and integrate our understanding” or something like that - but I did rankle at a “trivial” proof which is invalid due to equivocating between multiple definitions of the word “interesting” (which should really take an object, “interesting to me” vs “interesting to something smarter than me”).
Interesting is a bit in the eye of the beholder. Some people probably find maths boring full stop, some probably find all of it interesting.
I think it really depends on what the universe looks like as you drill down into it. It seems like the further down into smaller systems you get, the more analytically complex it gets. And then there will always be more value in enhancing the generalisations you have.
I would argue that novel and/or valuable results are not necessarily interesting!
I (a human) am interested in things that are applicable to my realm of understanding, but I see a very plausible future where novel and/or valuable results leave that realm.
I'd further argue that's already the case for most math for most humans. What's interesting to Terrance Tao is rarely of immediate interesting to me.
That's a good question that can be answered by methods in mathematics.
> we won’t have enough people for all the jobs that need to be done
Who defines jobs that need to be done? Businesses and institudes do.
There is no magical entity that observes the need for jobs and creates them at ideal rate.
Consider a world with one person in it. That person defines the jobs to be done. As soon as that person is disabled for one reason or another and is dying due to lack of care, there are by definition more jobs to be done than people to do them. "Businesses" and "institutes" are just constructs that exist to summarize those needs at scale.
I would say aggregate human desires determines what jobs need to be done businesses come about to fulfill the gap.
Yes, I think it would be viable if it wasn't corporations reaping all the benefits (income) from AI right now.
I find Tao's writing on this very helpful, particularly because a lot of the ideas generalise to most knowledge work. It is darkly fortunate that some of the smartest and most articulate people on the planet are affected by the AI race - it gives me a little hope that alignment and control can be solved.
(this is not from Terrence Tao, he just reposted it in his blog)
Godel's Incompleteness Theorem and Tarski’s Undefinability theorem ensures an ever expanding space for proving true statements.
Neither LLM's nor humans can exhaust it. So yes, both mathematicians and LLMs are needed.
Both can contribute and there will still be work leftover.
We need human mathematicians for humans.
But companies are our gods (don't believe? E.g. companies cannot die from natural causes). They don't need puny humans. They need AI.
Companies are nothing other than the tool of rich people used to control normal people. There's some sort of mass delusion that "companies" are evil. Guess what, a company has no own agency to destroy the environment, exploit people, profit from suffering, etc. It's just following the whims of its shareholders, who are humans with accountability.
Humans still excel at posing the right questions and finding truly novel, insightful proofs. AI just brute-forces.
AI does NOT “brute-force”, else we could have had those math breakthroughs decades ago. Seeing this dishonest goalpost-moving every few months is so tiring.
> Similar logic applies to every industry and every job. And it comes to the conclusion that we won’t have enough people for all the jobs that need to be done.
In the (extremely) short run, yes. In the long run, those jobs will also be done by AI.
It's like chimpanzees seeing human society and saying "look how complex it is, imagine how many chimpanzees it needs to maintain it".
> In the (extremely) short run, yes. In the long run, those jobs will also be done by AI.
Let's be honest: we don't know. Maybe you're right, but for the moment it's more likely that you're not. And countries cannot bet on that vague intuition at the cost of destroying their research communities and world leadership (which takes decades if not a century to achieve).
I don't think research communities should be destroyed.
Best case, they still matter.
Worst case, AI kills us all and it's irrelevant that we "wasted" money on research.
Chimanzee vs Human is not an appropriate analogy. First, because they cannot communicate properly.
A better analogy: look at all these highly trained engineers, mathematicians, doctors, writers, philosophers, writers, scientists.
How many dumbass politicians do we need to keep it all running smoothly?
Turns out no matter how dumb politicians were, overall society has been developing positively over the history of mankind.
Math proofs are special because they’re verifiable, safe, and do not require physical experiments. You can perform exhaustive parallel search in simulation with RLVR.
Do you think this applies to say, surgery, as well? There are few useful problems that share these properties.
Do you think surgeons start performing surgery for the first time on people?
Sounds like you're saying that so long as the LLM with robot arms starts by operating on mice, cows and pigs you're fine with it
wtf are you on. Surgeons practice on much more than animals.
Not all of it would translate, but robotic surgery could be done on anesthetized animals in a mass RLVR way.
This is wrong, both morally and technically.
First off, I can’t imagine anything more torment nexus-y than throwing billions to automate and scale the torture of animals. If each token is a “cut”, how much suffering does 10 trillion training tokens (lower bound) corresponds to?
Second, this still doesn’t cover all the properties that make math proofs doable. It requires working in the physical world. You must physically capture or grow 10T cuts worth of animals. You cannot verify success so easily, either. Cancer cells, for example, could regrow over months. You would need to keep the animal alive and regularly test the animal, which would be difficult to scale. And most people wouldn’t trust the world’s greatest vet to operate on them, anyways.
I'm not advocating for it, I don't know enough. But we already slaughter lots of animals and raise them in horrible conditions, and all robot assisted surgery already goes through animal trials.
> Second, this still doesn’t cover all the properties that make math proofs doable. It requires working in the physical world. You must physically capture or grow 10T cuts worth of animals.
It would likely be sim2real with that as the post training, reducing that requirement a lot.
> Cancer cells, for example, could regrow over months.
In that specific scenario you would likely train on receiving no unrelated injuries during the surgery, and have induced conditions with stuff tagged molecularly that you can then verify efficacy from without waiting months.
Depending on how much more data efficient sim2real makes it, you could end up seeing companies pushing it only for actual procedures the animals need but economically would never get; botched surgery and the animal gets euthanized before waking up, which they could argue was already going to happen.
> And most people wouldn’t trust the world’s greatest vet to operate on them, anyways.
Robotic surgery systems, already go through animal trials before being used on humans. So do many purely human surgery techniques.
> robotic surgery could be done on anesthetized animals in a mass RLVR way.
well that's an awful image
But we're not building an AI society - to what end?
We're building tools to serve the human society.
A few people with way to much power are building AI to achieve their utopian, sci-fi dream. A few other powerful people are using that first group to achieve their post-democratic goals. It’s the most obvious top down attempt at imposing their vision onto the rest of the world. They aren’t building tools to serve anyone else than themselves
>A few people with way to much power are building AI to achieve their utopian, sci-fi dream.
You mean dystopian. If it were a utopia everyone would be happy to welcome the new world order.
No, I mean utopia. For them. A dystopia is what we will all experience, when the utopia they are pitching fails to materialize. I’m using the traditional, correct definition of the term, where a dystopia is a utopia that failed to achieve a paradise for the people and instead becomes hell
utopian for them, dystopian for us
There's one flaw in the evidence for the logic chain. The hugging face attack is used to demonstrate three things: the need for oversight today (fundamental to the article) and to demonstrate some kind of drift or unexpected capability gain, and finally to hint and some fundamental morality of the AI or at least the risk of drift from our morality.
I'd argue that what the hugging face attack illustrates is that large AI companies are motivated to have bombastic claims supported by bombastic demos. The model was clearly trained or encouraged to work as it did, as evidenced by the fact it keeps using this particular escape hatch.
And the fact that it aligns with prior and current calls for what very likely might be a regulatory capture / oversight capture move right before IPO. It aligns so well with this "barely constrained superweapon" narrative it might as well be PR.
How does your logic explain that OpenAI seems to want to hide the extent of the HuggingFace incident, and every week we learn from 3rd parties about new victims of the hack?
4D chess? They want others to find the hacked services, so the report of how dangerous the agents are seems more "legit"?
IMHO hiding details of the hack helps conceal that they built it to do what it did, and maybe were able to know it was working just as intended.
The fact it keeps doing it, with more and more evidence, is a sign that it's built that way.
This is a program running on their montoroed machines that they purpose built and monitored its training at every step. I think it'd be way more suprising that they didn't know it used note taking and cross-run memory.
We do not.
AI is not AGI right now, it can't think or come up with something new like a human brain. It still uses knowledge that was made by a human and was published on the Internet.
Please formulate and relate this to a concrete prediction of something that a current frontier AI cannot do (that a human can).
There is no evidence to substantiate what you are saying.
Every possible proof exists already as a possible generation in the grammar of lean or rocq. In no way does that mean we have discovered everything.
Is there a proof that every possible proof is a generation in the grammar of lean or rocq? Sounds like an unsubstantiated claim.
You're confusing the mechanical language of math with math itself.
The map is not the territory, etc. If math was just an elaborate linguistic Glass Bead Game then we wouldn't be funding it. The intuition is that the surface rules of math help uncover the underlying structure of reality.
Why is Terry Tao even talking these arguments when all OpenAI did was a sophisticated brute force search with LLMs?
Give them Navier-Stokes in a vacuum
Same reason Gary Kasparov and Lee Sedol started their morning in one universe and ended their evening in another.
Brute Force or not, the game is being played and won without them all of a sudden, and even worse: at a level of output far beyond them
What do you think all the PhD's and post-doc students are doing...? It's not called graduate-descent for nothing :p
> What do you think all the PhD's and post-doc students are doing...?
Thinking.
The article wasn’t written by Tao.
Because it’s high time to rethink the incentives and rewards structures to nurture the next generations of mathematicians when corporate money and automated AI systems are mining the pool of limited interesting problems without giving back or enhance our understanding.
The problem they care about is not just “AI replaces human mathematicians” but rather the society thinks mathematicians can be replaced by AI.
Brute-force search is called that specifically for its lack of sophistication. A "sophisticated brute search" is an oxymoron.
> We (humans) should help humanity flourish.
I think this axiom is of course true. But the mistake the article makes, in my opinion, is to try to apply this axiom separately to each domain. If we have this as the over-arching axiom, it is not clear at all that humans should be steering the development of mathematics. Maybe it would be better for humanity if the department of world math is run by AI.
What's up with these insanely off base articles on HN this morning? Every single one of these is an unhinged pro-AI anti-human schlock piece.
You're talking about this article? I don't see how you could get that, fundamentally I read it the other way around. Unless you're talking about the base assumption of capabilities.
HN is now heavily astroturfed or overrun by bots (or, alternately, low-quality posters now indistinguishable from the previous), and this is especially so in AI-related threads.
One tell is that most comments barely exceed one or two sentences (because otherwise AI detection gets easier and much more reliable), when this was not as much the case many years ago. The drive-by comments are also low / zero quality, mostly expressing a feeling or agreement/disagreement, and primarily driven by ideology or pre-existing beliefs and commitments.
Look at non-AI-related threads and you'll notice a large distribution shift relative to AI-related ones.
EDIT: Basically HN is orange Plebbit now. If you doubt this, compare HN discussions to those on e.g. lobste.rs, LessWrong, The Motte, DSL, ACX, or other old obscure forums. You'll notice those places have their own very serious biases and problems, but it is obvious the vast majority of posters are nevertheless human and making some minimal efforts.
Now compare Reddit and 2026 HN to the above, and see if you can confidently say the same.
My HN frontpage is filled with anti-AI rants
Friendly warning to those who might not be aware: the vast majority of comments below posts like the above will be left by (otherwise intelligent) programmers who think mathematics is a closed system where one attempts to solve endless Olympiad-type problems. I wouldn’t take any of it seriously at all. Better to listen to what those who actually know what the subject is about have to say.
Unfortunately, mathematics (especially pure mathematics) is by its very nature very, very poorly understood by those who haven’t worked as a mathematician. Even worse, those who don’t understand are seemingly not at all aware of their misunderstanding and are entirely confident in their (very wrong) characterisation of the subject.
I did systems administration for a university math department for several years. I came to the conclusion that mathematics (and perhaps philosophy) were both topics where it was likely that no staff members in that department could describe "what goes on here" and that possibly even within the department, one professor may not be able to describe what another professor's actually doing.
The closest I could come to describing math is "some abstract process where imagined structures are characterized and extended; the most critical part of the process is identifying where seemingly independent structures are found to actually be fungible in some previously undiscovered way".
A simple example is
The clue is burried in the text of OP's article.
> Mathematicians can also consider wholly redirecting their skill sets to work on real world problems. I’ve actually been encouraging mathematicians to consider thinking about working on government or other large-scale societal issues.
The fact that this is a radical departure from the norm is part of why mathematics (and philosophy) is often seen as some intangible or ungrokable science to many outsiders, as they're generally approaching it from a perspective of "Okay, but why, what is this useful for?", and the answer "For the science of it" doesn't tend to land with people that aren't already passionate about said science/discipline and are just trying to figure out what it even is or involves.
Doesn't help that there is a pervasive sentiment in American Academia (not sure about elsewhere) about Math being *the* hard science, and I mean hard as in difficulty, so a lot of people get intimidated by it before they ever give it a chance very early on in their academic life and carry that through the rest of their education.
So there ends up being a rather small pool of people that are in(to) the field, and rather high friction for stimualting interest in it from outsiders from the way that it's taught, and a massive difference in the perspective of it's use between it's diaspora and the unmathed masses.
Mathematicians who do work on "real world" problems are mostly doing so with theoretical physicists, genomicists, and cryptographers. None of these count as what people consider valuable other than because they are hard.
That said, few 50 (or even 40) years ago would have predicted that completely abstract number theoretical computations about primes, discrete logarithms, and elliptic curves would be the foundation of our monetary system.
> I did systems administration for a university math department for several years. I came to the conclusion that mathematics (and perhaps philosophy) were both topics where it was likely that no staff members in that department could describe "what goes on here" and that possibly even within the department, one professor may not be able to describe what another professor's actually doing.
And this is indeed why it is not going to be taken seriously as an academic or (more importantly) an economic endeavour done by humans anymore.
That won't stop the career mathematicians from protesting and having a cry here trying to justify themselves.
the author right in that post pitches his ideas as generalizable way beyond math, so why the snobbery?
Could you clue us in? What is mathematics actually about?
Good question, and maybe I should have provided an alternative description rather than just criticising.
Unfortunately, it is actually surprisingly hard to pin down, and I think mathematicians (and, as a student, I count myself as one to some degree at least) now have the task of making this a lot clearer. If we want to justify our existence in the face of new machines that can seemingly ‘do our work for us’ (so far in a restricted context), we should give a robust defence of our practice. If we can’t do this, we simply don’t deserve the funding (which, by the way, again contrary to some misguided statements here, isn’t very much anyway!). I think all of this will become clearer to outsiders as time passes, but for now it’s not easy to give a quick answer — though I can try.
Mathematics is about understanding things. Isn’t that what every subject is about? Well, I suppose so, but mathematics more specifically does something like the following:
(1) observe some phenomenon in ‘reality’.
(2) attempt to formalise that phenomenon in such a way that it can be manipulated purely symbolically.
(3) use this (perhaps fairly arbitrary; remember that we can invent as many formal systems as we like) system to deduce from our initial assumptions new facts that would otherwise have been very non-obvious.
It seems like outsiders have a decent grasp of (3) and the application of AI to it, but have very little idea about the other two steps. It seems to be widely assumed among non-mathematicians that problems are essentially god given and that the job of a mathematician is therefore to chug away on these problems, manipulating symbols and trying out tools, in the hope of learning a yes/no answer to each one.
The first two steps are by far the hardest and most important, and they’re also the parts that AI seems currently unable to help with.
NOTE: this is not a deeply insightful description of what the subject is about, and there are many better characterisations out there. I think Tao and various others have written recently about why complicated and inscrutable AI-generated proofs aren’t nearly as valuable as one might imagine. (That’s not to say there’s no value to such proofs; perhaps in time, as technology improves, mathematicians will come to accept AI as part of the process.)
If you want to understand all of this issues better, reading the recent slew of guest posts on Tao’s blog would be a very good start.
Despite loving Mathematics and having considered being a Mathematician myself, that wasn't a very robust defense. Also, even though I'm a big Terence Tao fan, I'm not sure he has sorted that out this defense in full himself.
This post which he forwarded was quite poor in my opinion. Confusing, all over the place with AI criticisms and promotion of the AI hazing being done by mathematicians.
X thousand mathematicians who want to protect their livelihoods signed a bunch of letters against AI. Duh. We've seen similar movements from every profession that has been displaced ever.
Terence Tao uses AI and has made a few good points on how to use it. But defensiveness leaks into almost every defense of the role of humans in Mathematics that I've read, even his own at times.
To be clear, I actually believe that Mathematicians aren't going away, but I dont have enough knowledge about the life of a professional mathematician to articulate a path forward.
This "path forward" is what I'd like to see. We need a top mathematician with enough intellectual honesty (Terence Tao qualifies, I think) to start this questioning with "there's actually no role for human Mathematicians" as one of the options on the table and go from there.
Though your points above may all be valid, you're asking us to just accept your view that the parent's wasn't a very robust defense.
Can you provide an explanation for why their claim that (1) and (2) are the hardest parts is false?
Or perhaps provide an alternative definition of Mathematics that is more explanatory than the one they've provided?
For me, mathematicians seem to be still having an inner discussion rather a making these essays for the more general public. In any case, I think that statements of the type "there's actually no role for human Mathematicians" are completely non-serious, so it'd sad that the discussion concentrates on that.
I agree, but it's still something we shouldn't eliminate a priori. I think Mathematics as a profession will be greatly enhanced by AI, but I can't prove that.
There’s a simple point to be made: if no one in the world understands an AI proof, what’s the point?
Maybe another AI could use the proof.
How would we know it was correct?
If you feed an AI nonsense in its training data, it will generate nonsense
My understanding when I was practicing is that the trend in modern mathematics is to focus on spaces with a certain kind of structure, and maps between those spaces that preserve it, and then what are invariants are preserved by those maps. Structure-preserving maps between categories of such spaces - "functors" in the language of category theory - are especially neat.
That's certainly different from Olympiad-style problems.
Pure mathematics reduces to abstract "relationships" and their implications largely.
To know what something is about, a natural way is to do it yourself.
Wow, I'm glad AI is going to start humbling people.
> To know what something is about, a natural way is to do it yourself.
literally what software engineers were doing for decades though
software is mostly just simple math, for the most part, until you need to do something more complex for some hairy algos lol
Actually, if you were to convert most software into mathematical notation it would look incomprehensible. We are not interested in gatekeeping as much by making things look more convoluted than they need to.
This is a great point. When I was learning machine learning in the early 00s, there were many papers where I would struggle to understand exactly what the mathematics was trying to communicate, but where I would look at the matlab source code and say to myself "that's all?"
Wondering about the world, using the simplified language of "mathematical logic"
A lot of programmers consider themselves to be applied mathematicians.
Maybe more in years past when Comp Sci was a subset of Math Departments.
In my observation, most programmers and engineers lose their math chops over time. The math they need is mostly baked into their tools, such as CAD. If a problem requires more advanced math, it's given to a "math person" in the department. Often, the "math person" is also not allowed to touch the production code.
i'd be on board with this concept but the author seems to believe his point is generalisable to all under industries, hence committing the same fallacy you're talking about at a large scale.
but you're still right. i disregarded his take, as you would with mine re. math.
.
I’m just shocked by how “intelligent” people believe a token guessing system can be
I am shocked by how intelligent people seem.to believe in some form of dualism or supernatural element to thought and intelligence.
I am shocked how people can deny that solving Navier Stokes requires some sort of intelligence. Even Doctorow talks of "brute-forcing" a solution. Brute-forcing leads to combinatorial explosion, so there must be something more going on here. Otherwise you could just put this problem into an automated theorem prover (we've had those forever, they are actually just brute-forcing it).
> Otherwise you could just put this problem into an automated theorem prover (we've had those forever, they are actually just brute-forcing it).
There are many automatic theorem provers that do very clever stuff, just as the underlying theroy describes.
> I am shocked how people can deny that solving Navier Stokes requires some sort of intelligence.
It is absurd to waste time discussing whether it is inteligent or not. It is just an algorithm, we know how it works, and it does exactly what we expect it to do. LLMs are not magical things. The main difference is the scale: for Navier-Stokes they spent in 3 days more money that the whole mathematical community over the last 20 years easily.
By the way, I'm not saying that LLM's are useless, that I'm anti-AI or anything like that.
Intelligence does not seem to be magic either, as LLMs are proving now. It is indeed a waste of time to argue that LLMs are not intelligent in their own way, they obviously are. If Navier Stokes doesn't convince you, nothing will.
I just used a £89 Codex subscription to do very intelligent things with it, stuff that I would have had to sit down and ponder and work on for quite a while, and I have a PhD in that. I didn't need to do anything special except explaining the problem(s) to the AI, and my theory of it so far. It took it from there. If that is not intelligence, nothing is.
Pick one:
- solving "frontier" math problems requires intelligence (by human or AI)
- solving "frontier" math problems is dumb statistical prediction of next token (by human or AI)
Turns out what you call "intelligence" was never needed to do math.
It was needed before the LLMs and training data existed in digital form. Euclid, Gauss, Turing didn't have that benefit.
'There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders control ... to the less capable species'.
True with genuine species. But we should take note of the plentiful counter examples within human societies. How about politicians and our method of choosing those people to whom we delegate the most critical decisions regarding our and the Earth's future? We select politicians mostly either by rote or via their persuasive rhetoric, their general personality & likeability and probably least of all by their intellectual capability or indeed general capability in too many cases. That is not to say intellectuals are necessarily any better at the job. There are numerous other examples in human organizations as we know, sometimes to our cost. Truth is that we cannot even agree on how, as a species together with the other life forms on Earth, we can all 'flourish' though there are lots of great examples working in local environments.
Funny... for me, this article was right next to the link to this current thread.
AI chatbots give wrong answers to financial queries 'most of the time' (ft.com) // https://www.ft.com/content/c0cd359d-df84-4208-a789-ffa864b43...
This sounds like a student in algebra class asking, "when will I ever use this in my life?"
Here is a link to the original article:
https://poshenloh.com/posts/20260919-math-ai
The original posted link from OP is from Terry Tao’s website where the article was posted as a guest blog post.
It's astonishing how much more readable Tao's version of it is.
That's what I thought also, that is why I posted Tao's version.
Because the AI needs new stuff to train on?
I sometimes think about this: to grasp the fundamentals of knowledge and complex phenomena, perhaps we need an external system rather than human knowledge systems. By that logic, maybe we need AI, which can handle far greater complexity.
Human capability, when you think about it, is complex. Why is Newton praised as being so damn great? He established the law of universal gravitation, F=ma. Why is that such a big deal?
He distilled countless phenomena in an open system into a single mathematical formula.
What makes it great is that he found common state variables and relationships across entirely different phenomena like falling objects, planetary motion, collisions, and artillery trajectories.
But does F=ma hold true for the entire macroscopic world? No. There are various conditions and specific situations in motion, but within most scenarios and a certain range of approximation, it outputs values that are useful to humans.
Why is the Schrödinger equation so great? Because it turned the time evolution of quantum states into a calculable mathematical law.
Human thought is essentially creating a closed system by deciding what to cut out and what to keep from the infinite degrees of freedom in reality. Academia is what reinforces that closed system.
A great theory is great not because it perfectly replicates reality, but because it compresses the immense complexity of reality into a small, closed formal system while still managing to explain a multitude of phenomena.
In that process, it feels like human thought and progress are shifting into a different framework. What LLMs do well is primarily exploring within the ontology and representation space that humans have already built.
I think there are two broad categories of discovery: One is forming a new closed system, and the other is connecting fragmented knowledge within that closed system. I feel that the vast majority of research focuses on the latter.
What LLMs excel at is finding unvisited points within a given representation space. This is typically the process through which master's and PhD students connect dots, build their skills, and form their own mental models. But the logic behind criticizing LLMs seems to be that they eliminate the very work these graduate students need to do in order to grow.
However, looking at it from another angle, perhaps our current knowledge systems and classifications have reached a limit, suggesting that we might actually need a completely new classification and knowledge system.
What is the core principle of an LLM? It's predicting the probability of the next sequence.
Let's say you type the word "cat". Cat - is cute (90%), want to eat it (6%), furry (4%). Because "is cute" has the highest probability, the next sequence proceeds in that direction.
Within this framework, human knowledge and logic largely operate the same way. Once an initial logical proposition is established, we follow it up with whatever makes logical sense next. From that perspective, I think LLMs will actually do this better.
But what is it that LLMs cannot do right now? They cannot create that initial logical proposition. I believe they lack the ability to carve out a closed system from an open system.
Stacking logic step-by-step within a closed system—LLMs do this exceptionally well. But whether that constitutes true "intelligence" is a different matter.
I feel that being logical does not necessarily equate to having intelligence.
Humans preserve and create different mental models and knowledge systems within an open system. Just as your thoughts differ from mine, LLMs lack the ability to form these distinct mental models.
If so, within these limits, what humans must ultimately do is construct the logical frameworks that LLMs can then fill in. Perhaps a new kind of logic dedicated to designing these frameworks will become the next major trend.
Viewed from this perspective, I have no idea if we are in a mere technological transition or something else entirely. Or whether it is even correct to say humans are strictly necessary to build that framework. Maybe my learning is just lacking.
Why do we need any job? Check "Bullshit Jobs" by David Graeber. Not saying being a mathematician is a bullshit-kinda job, but there have been so many made up positions way before this AI-era. In that sense AI isn't changing much.
> Why do we need any job?
If not for nothing else but to form a basis for how to distribute/share/hoard the wealth created by a society. The eternal question - who gets what.
Most people work for a living. Then there are people who find meaning in work. And people who advance in life by working. You want to do away with all that?
waiting for the xkcd for that
I always think of this one but I bet there's better
* https://m.xkcd.com/435/
Got to be philosophical about it all.
Pure math has been overvalued for a long time. Now that we don't need humans for it, it's really time for pure mathematicians to pack it in.
Who says we don't need humans for it? Are non-mathematicians going to be guiding agents to solving pure mathematical problems?
Well it's actually nice, maybe more people will be able to do world class math with the help of these tools. There are few fields as elitist and hostile as pure mathematics, most mathematicians I know build their whole life around their profession and their self esteem is strongly coupled to the fact that they can do things that most other people can't. Naturally, many will be devastated when (if) you take that away from them. That said I think AI is still overhyped and human mathematicians can easily outthink it in most domains, look at how difficult it is for an AI to write even a single decent paper, a good PhD student can easily outclass it in that regard. All of these impressive results were generated by having world-class mathematicians steer the systems using highly tuned prompts, so I see it more like a super violin that produces beautiful music when played by master violinists rather than being a fully autonomous orchestra which many people are led to believe this already is.
And maybe let's not only hear the opinion of two or three Fields level mathematicians with blogs, 99 % of the worlds mathematicians in academia might profit from these tools as they might partially close the gap between them and the world elite, making creativity and tenaciousness more important than having the right neocortical structure allowing you to outperform 99.9 % of other humans at keeping context in your head and making predictions, AI can do that better now with the right prompts.
> There are few fields as elitist and hostile as pure mathematics
Is that so ? Sounds hyperbolic.