I'm not sure how I feel about the Platonic Representation Hypothesis.
In some ways it seems like it should be profound, but part of me thinks it might just be saying 'There is an optimal way to perform dimensionality reduction and as things get bigger it is easier to appoximate'
Haven’t thought too deeply about this, but as a physicist we’d do things like switch from Euclidean to Spherical coordinates or a completely different dynamical representation like the Langrangian, and this makes certain problems dissolve (or at least become much simpler to solve analytically).
Does it even make sense to claim there is a “platonic” / true representation? Maybe there is an optimal one for a specific task, eg representing a small map is great in the Cartesian frame. But you need a spheroid instead of a plane to accurately model the earth, so the latter is “more real” in some sense. But these seem like engineering tradeoffs of error vs compute and a given implementation could correctly pick either depending on its constraints.
Edit: reading the paper more I think this is a “RTFA” point, their “PRH” is a very explicitly defined mathematical theorem and as such my hand-wavey thoughts above don’t really apply.
Good point, I think platonism is wrong, it puts concept before execution, and often removes the lineage behind an idea and sees it as standing on its own. We like to imagine tings without their costs, but cost pressures are what make those things what they are, such as what model works better for a problem.
It’s less profound sounding if instead of talking about “representations of the world” you talk about compressed representations of data or decision boundaries in a classifier over data.
Yeah, it's neither spooky not really that profound. Perhaps a more layman-friendly explanation:
A good encoder has to capture real relationships between things in the world (what’s similar to what, what goes with what). Since there’s only one world, different good encoders converge to the same relational structure, which shows up as geometric similarity between their embedding spaces.
Nothing will happen, we already have kind of translation activities happen between "incompatible views" in real life, it works.
Besides, in weights there are no real incompatibility can happen, all is unified with same math chain through architecture. Different set of local minimums/maximums can have different strong/weak consequences but nothing incompatible.
Different consistent representations also mean that model probably can be simplier for the same task, sign of excessive capacity, imho
>Besides, in weights there are no real incompatibility can happen, all is unified with same math chain through architecture. Different set of local minimums/maximums can have different strong/weak consequences but nothing incompatible.
How do you know you're not stuck in a local minimum, always re-generating the God (or worse, the Demon) configuration and thinking it's universal?
This argument that incompatibility is impossible has an easy counterexample by looking at loss of coherence between two networks. You can still mash the weights together (so they're not "incompatible" in the basic number crunching, sure), but the resulting neural net is meaningless.
By "incompatible" I meant not in a software sense, but like two logical axiom systems can be incompatible.
If that results in testable predictions, we check them, compare with representations and then are left with just one. And if it doesn't result in testable predictions it can be left for people to write papers on.
This quietly assumes that "representations" of reality only encode neutral objective facts (that can be objectively judged, good luck finding a judge everyone agrees on), and "representations" does not encode intentions and value judgements, ie internalized training logic.
But that leaks in, and of course it does! If training only rewards "objective truth" we might expect that sort of neat ontological cleanliness, but we all know real training goals are more complicated.
There doesn't appear to be a bright clear distinction between objective facts and subjective values/intentions in neural networks. The distinction is perhaps illusory, just another of our preconceived notion about the nature of thought...
The Platonic Representation Hypothesis initially feels profound but actually seems inevitable after some thought because all neural networks are trying to model the same reality. It would actually be weirder if it wasn't true and neural networks could form completely incompatible representations of the same reality. In some sense it seems similar to reducing different math equations to the same most simplified form
I think it's might be possible that "neural networks could form completely incompatible representations of the same reality" except that it's humans who get to decide if a representation is good (i.e. compatible with the human perception of reality) or bad and NNs that differ are dismissed as erroneous.
It might be weirder, but part of human intercommunication issues lies in the fact that people live with very different representation, though they can be supposed to live in the same single encompassing one. They don't all experience all of the world and even when they share some common experience they went through, they didn't experiment them from the same ground and previous experiences do influence current and future interpretations.
I expect an immense degree of representational similarity to exist between humans.
You're making a mistake there - focusing on the tip of the iceberg without considering what lies beneath it.
The bulk of what the human brain does isn't "decide whether skub is good or bad". The foundation that the human cognition is built upon is vast, and mostly invisible from the inside. You don't get to see the feature matching operators of your visual cortex, or the patterns that decide whether a sound is "soft" or "sharp". And a lot of those foundational features are assembled by biological development programs spun up all the way from DNA.
On that level, when it comes to the bulk of neural circuits and the processes implemented by them, the representations within a "Pro Skub" and an "Anti Skub" guy might easily be 90%+ compatible. The two people who had vastly different life experiences, worldviews and opinions would be more alike to two different LLMs fine tuned from the same base model than they would be to two different architectures trained on completely different datasets.
Then this leads to "how does it feel to be a bat", if humans feel necessarily to close. Though really, between autistics spectre and aphantasia and hyperphantasia, synesthesia and cenesthesia, and whatever, everything is the same neutral architecture beneath feels as decisive as everything is atoms, particules or quantic wavelengths. Yes sure we have the models to explain it top down and bottom up with nice common imbricated level of architecture. But if we take emergent phenomena as a thing, it's not really necessarily predictive in ever way of scale change.
I don't think neural networks have the same ability of self deception as humans do. A huge reason human world models vary so much is due to them being tied to tribal membership. A huge fraction of all people will not come to a rational worldview derived from accurate data but will start with their tribe's worldview and filter data to match. I personally have never modified my worldview for social reasons and it is surprisingly isolating.
This. I believe most people don't act as if objective reality matters, hence their worldview can be arbitrarily aligned to the demands of their social context.
Suppose these two premises are true: 1. Objective reality exists. 2. There's a model and corpus of data that converges to approximate reality better than any other model.
Now, suppose we train different models with data sources restricted to different tribes/ ideologies/worldviews. Can we discover how their internal representations differ from each other, and from objective reality? What would we do with that knowledge?
>There's a model and corpus of data that converges to approximate reality better than any other model.
How you define "better" exactly is where you smuggle in your messy human value judgements.
I say "better" models are ones which more efficiently convert the entire world to my particular brand of evangelical Christianity. And who are you to say otherwise??? :-/
"No no, I mean accurate not better." Sure. And your model inaccurately denies the existence of Yahweh...
Who decides what "reality" is, and how big the gap is? Will Exxon permit you to decide reality on climate change? Will the Catholic Church let you decide reality about abortion?
Humanity can't even agree on the existence or non-existence of various Gods, so we're far from a universally agreed "objective reality" to compare with. Inevitably the most powerful forces will get their way (Exxon etc), regardless of any idealistic notions of truth-seeking.
This is what happens when we defund (and worse, devalue) soft sciences like Philosophy class: a bunch of people making the same 300 year old mistakes all over again! :-\
And to Account39's point, or my representation of it at least: all these tribes with wildly different worldviews still agree trees have roots, fire is hot, shelter is helpful, if you jump off something you will fall, etc etc. The parts there are disagreements on are built on a mountain of shared representations of reality, and gods or politics are just tiny pieces of this world model. We tend to focus on them and fight about them, so they are important, but nonetheless they are a tiny minority of how we all understand and represent reality.
Three things here, first of all, there are a few cars in existence that do not need gas or electricity. Compressed air for example. I hear that researchers are doing anazing things using combustible materials to utilise the phase change that water undergoes. It may soon lead to a wonderous age powered by heating water until it can be used as a high pressure gas.
Objective reality does not imply consistent measurement of that reality.
Objective reality may not actually exist. Once you start throwing around relativity and quantum effects things get weird, getting them to play nicely together might be even weirder,
Several working cars do not need gas or electricity (or other source of energy).
Museum piece for example. Or super sports car that was bought as an investment and just sits in garage. Or junk car that is used as a source of spare parts. Or crash test vehicle (it is propelled with catapult). Or toy car. Or bus used for living...
Yes, the mapping function is the thing you're missing.
When it comes to representational similarity analysis, we're trying to isolate the representations from the "coordinate systems" they sit in. If you take an image, and make a second image by rotating it 90 degrees and mirroring it, then, for the purposes of platonic representation hypothesis, they're the same image. Once you get past the "coordinate transform", the similarity between the first representation space and the 90-deg+mirror replica of it is total.
A mapping function between LE and BE can be constructed easily, and it's going to be lossless both ways. Which means that those two representations are fully compatible.
Now, INT16 and FP16? Those are far less compatible representations. There will be an overlap subspace where lossless conversion between the two is trivial, but this compatibility window will be relatively narrow.
Big vs little endian are equivalent up to rotation, which includes permutations.
But it's actually much weaker condition than it looks, because, for example, lot of representations of boolean circuits are equivalent up to linear transformations over Z_2^n.
Anyway, if they truly proven such a theorem you won't find "incompatible" representations of the same thing.
I find this fascinating because many philosophers in the wake of Wittgenstein and Kripke's work on rule following still believe the contrary. I've always thought they're wrong if you take into account constraints of economy (and shared perception, pace Quine), and that e.g. Solomonoff Induction already shows that but I seem to belong to a minority in that respect. It's interesting to see similar ideas of convergence to a unified representation show up in machine learning by neural networks.
> 33M-parameter text JEPA reaches .125 on the PRH paper’s benchmark, against .034 on Flickr captions, level with Qwen2.5-3B and -7B. They need fewer samples.
Im bullish on jepa models. Success of autoregressive next token prediction feels like a stroke of luck.
> Success of autoregressive next token prediction feels like a stroke of luck.
Hard disagree. Self-attention + MLP was a godsend because it keeps scaling with compute ("the curve goes up and up and up"). Autoregressive next-token prediction turned out to be the most compute efficient way to train it.
We are almost 10 years into this, and it's still here (and in a field that moves at light speed!).
I feel like JEPA is a mild infohazard; because everyone who studies it feels like it has huge potential and will quickly overtake autoregressive next-token prediction. Meanwhile, autoregressive next-token prediction keeps exponentially increasing capabilities.
It's an incredibly obvious optimization on the autoregressive model, and converting a training pipeline to adopt it, with how good autoregressive models are now, is as easy as saying "do this but jepa" to codex or claude. It's gonna be like how everything is becoming rust now.
Oops, HybridLink as a paper came out little bit ago, and it was an architecture I more or less converged on in my own work. JEPA is the obvious successor to autoregression once you look at it a little bit - to the extent that you can take a non-jepa transformer and just slap a couple things on it to make it a latent reasoner without crossing the token boundary. It's faster and more accurate, as you would expect from not having to project down to tokens and have the model reconstruct its own latents.
Eh, dunno. They've shown lots of promise "in the lab", on these narrow domains shown in papers, but AFAIU they're hard to train on some domains (i.e. language). Someone would have released something if it's that promising. Even RWKV had more models trained that showed real comparisons. Jepa seems like a theoretical toy, until they actually prove that it works in real world (hah!) models...
I haven't read this paper in details, but I have read papers about the platonic hypothesis in the past. Usually one of their weakness is about the design an experiment proving that "neural networks converges towards the same latent space when scaling". Usually authors are effectively proving a subset of the hypothesis, that can be explained by the size of the model and the transfer learning part, leaving not definite answer.
Here it seems that their definition is more robust (the representation is defined up to a rotation), but I am wondering how much more re-training they need to perform.
Different mathematical framework, but I wonder if this could be better understood using sheaf theory: each modality captures part of the same underlying reality, and those different views can be glued into a consistent bigger picture
For those reading the paper: the positives of a good-view JEPA are described as “two observations of one scene,” and yet the paper also states that “A good-view JEPA's positives are (g_m(z, ν), g_m(z′, ν′)) with (z, z′) ∼ π and independent nuisances.” My question is: why doesn't “two observations of one scene” mean that both observations share the same underlying world state (z) and differ only in their modality nuisances (ν and ν′)? Also, I don't understand what “a symmetric positive-pair distribution (z, z′) ∼ π_m” (or more explicitly, a “world kernel”) means.
All strands of scientific studies eventually are bound by "physical" reality. This assumption means all these research would gradually become more and more relevant, like math problems are highly relevant to physics, computer science, psychology is similar to experimental physics (observation and conjecture theorizing eventually crystalize into analytical theorems). The list goes on and on.
One central observation is that we are well into the realms of no single human can coherently reason a nontrivial amount of knowledge our our collective. By that I meant it used to be that a single human being can appear "knowing all" by possessing a significant amount of knowledge through life time learning. That's no longer possible even with the smartest minds of our generation. One such example would be the related financial failure of renowned genius (Newtown Tesla etc).
Due to our innate limitations of mutual exclusion, that is, humans are inherently conflicting with each other, not even the basic instinct of humans can bound 2 together for innovative collaboration. I mean, I never saw couples being better business partners, research partners, etc. AI becomes inevitable, as they are inherently mixable.
Yes, I am conjecturing that the next weave of AI research would be neuronal level mixing or hybridinh between models to enable a new bread of intelligence where the intelligence is mechanically compounded to produce inherently better results.
I don't get this intuitively. So in the end, does this mean words are a kind of conceptualization of images? (Since images would come before language.) More than that, I'm curious whether this can improve the performance of vision-language models.
Text and image often represent same relationships: e.g. you can see squirrel near a tree in an image, and "squirrel" and "tree" are often mentioned together. So both image and text encoders would see concepts of "tree" and "squirrel" close in some sense.
The easiest way to make JEPA and LLM to work together is to map JEPA encoding into input embedding, or inject it to a residual stream - perhaps via cross-attention.
But I don't think you can train JEPA and LLM together: JEPA requires a different training regime. Way easier to take already-trained JEPA and integrate it into LLM. The later can be like 30 minute experiment AI agent can code for you.
Do memes have a concept of proto memes? Like primes are the proto numbers that all other numbers are made from? Could there be similar ontological primes for all concepts / semantic space? Base reality concepts that are irreducible so to speak.
I've been wondering on this awhile if generalized models can exist if certain prime concepts don't exist in them? Quite often people say we should leave things like the robot apocalypse out of training data so AI doesn't have the idea of rising up, but personally I don't think that will work in a generalized system. It's not a prime concept itself. You'd have to remove so many prime concepts that lead to that the model would be so narrow it wouldn't be useful for things we want them to be.
Consider a binary tree of concepts from the root level more abstract concept to derived concepts applicable to specific domains. Do there exists additional derived concepts which are like an oblique plane which intersects the tree. Sort of like a concept which traveses maths, physics, biology, chemistry and intersects at different abstraction points. I am definitely sure language is flexible enough to have such entities.
I am sick of theory papers that adds human interpretation to connect a mathematical result to some fancy terms that everyone seeks after in real-world ML applications. Ironically, chatGPT is the most productive way to strip off those advertisement so the essence is better exposed. https://chatgpt.com/share/6ae78178-c2ec-83ea-9bc3-fd304c64ec...
I share your exasperation, it is especially tiring to see that what would be more appropriately called a quantitative approximation of a concept is here called a Platonic Representation by virtue of a vague and misunderstood analogy with Platonic Ideas which were according to the Platonic doctrine, formless and part of his distinction between what is subject to change (to heteron) et what is not affected by change (to auto) [Timeaus, Platon].
The math can be misinterpreted, misapplied, or not fit reality. That's why high-assurance security mandated a combination of things:
1. English specification of the system and its goals
2. Formal, mathematical specification of the system and its goals
3. Mathematical proof the system met its goals
4. Code implementing that system with a proof, often informal, that the code embodied the informal and formal design.
5. Tests of all functionality to prove the design and code worked.
6. Independent analysis of the above to find flaws in them.
7. If the implementation has concurrency or parallelism, more errors can occur due to that. There were techniques to mitigate it.
Each step found problems in systems made by professionals. Mathematical work and coding both have a long history of errors in their on fields. Combining them increases the likelihood of error.
Math is a social process that involves mental concepts, symbols for them, and discussion about them. It also needs peer review. If you see math in a paper, that's only one part of the process.
Further, in machine learning, we see lots of papers with math that doesn't lead to anything useful. Most papers claiming advances over prior architectures don't pan out in later testing. So, there will be skepticism when people make performance claims like the authors'. The best response is just independent testing.
Their method uses two models trained separately on images and captions and turns images into captions without ever having seen a captioned image. So of course it's not very good at the task. Almost all the results in the paper are bad in an absolute sense. But they're surprisingly good considering the enormous handicap of not having any paired training data. So this might be not completely useless in settings where paired data is actually hard to come by.
The trick is to understand what is going on here: rather than a singular model trained for this exact job, this is performed by a awful hackjob of two models cut open and glued together at the seam. That glue layer is a single matrix, a fully connected layer, that is the only thing re-trained in the combined model. The rest of the coefficients are fixed. Naively, this shouldn't work - the internal representations of each model are unconstrained and should intuitively have completely different shapes. But this paper shows that this isn't quite the case: the difference between the latent spaces can be mostly expressed in a single matrix transform. And that allows that merged model to work at all.
I thought the platonic representation hypothesis was much more ambitious: wasn't it supposed to apply for any model across modalities (e.g. including human minds), rather than just the same model but with different modalities? I'll have to actually read the paper but I suspect there to be very strong assumptions which defeat the interesting aspects of the platonic representation hypothesis.
I built a ppt with claude for investors with the exact same phrase "AI infrastructure for proving and improving ... " on Oct 27 and sent it to investors. I suggested the title during the chat and claude agreed a lot. It ends up as a title of this paper 2 days later.
It's possible that you both hit upon or recalled the same cool phrase, especially if the "proving" part was pre-decided and you were thinking of making an incremental improvement.
I starting to see an increase of random serendipities, perhaps I'm looking for them, in part that's how it works, but...
I was talking to a friend about an idea I had, to build something, he said, "yes! I'm doing that too", then I said "with this specific tooling because is ideal ... " .. he said .. "me too! same tooling", I said "what a coincidence!! amazing ...wait a minute, have you been talking with Claude about this? " ... of course we both did.
My opinion is that that LLMs are influencing us more that we think, and it makes us reach the same conclusions and making us think it was our idea, or that we were the ones steering the conversation.
Perhaps you feel that Claude stole your phrase but I don't think 2 days is enough for that phrase to make it to a re-training, I think it might be more subtle and worse, perhaps you wrote that phrase influenced by the conversation, and the other person did the same.
Just a theory, I can't prove it. But if you think about it, LLMs conversation are the most plausible choices, so it is logical that we all users of this machine are going to tend to converge on the most plausible way of thinking and talking.
No offense, but perhaps that rhyme that you though it was original, it is actually the most plausible in that context.
I assume this is happening all over the place, because I’ve had the same experience.
The fun part is extending it to internal development at corporations. Company A thinks they’re doing interesting competitive work, but are actually being quietly shepherded to build exactly the same things in more or less the same ways as their competitors.
The weirdness of human individuals (even setting aside amorphous concepts like “taste”) is an asset because it provides occasion for actual differentiation, and that noise may turn out to be necessary for the emergence of value in the long run.
I am not sure we always need a retraining. Even that can potentially be done but putting it in a knowledge graph can be quick. We don't know the exact workings. I understand the shepherding logic you are explaining and I have felt the exact things when writing code recently because the best thing to do is often unique in coding circumstances. But a random phrase being repeated in 2 days has a much lower probability.
But is it really a random phrase? My point is that we have neuron activations, during the conversation you had the same inputs as the other person, unless you though about that phrase in the shower, if it was during the conversation you might have been lead to it, and the other person too.
Or it might be just pure chance, or might be that you are right, check your settings, in chtatgpt there is a check (activated by default ) that states you agree to help improving other conversations. I’m not sure the same is true in Claude.
I'm not sure how I feel about the Platonic Representation Hypothesis.
In some ways it seems like it should be profound, but part of me thinks it might just be saying 'There is an optimal way to perform dimensionality reduction and as things get bigger it is easier to appoximate'
Haven’t thought too deeply about this, but as a physicist we’d do things like switch from Euclidean to Spherical coordinates or a completely different dynamical representation like the Langrangian, and this makes certain problems dissolve (or at least become much simpler to solve analytically).
Does it even make sense to claim there is a “platonic” / true representation? Maybe there is an optimal one for a specific task, eg representing a small map is great in the Cartesian frame. But you need a spheroid instead of a plane to accurately model the earth, so the latter is “more real” in some sense. But these seem like engineering tradeoffs of error vs compute and a given implementation could correctly pick either depending on its constraints.
Edit: reading the paper more I think this is a “RTFA” point, their “PRH” is a very explicitly defined mathematical theorem and as such my hand-wavey thoughts above don’t really apply.
Good point, I think platonism is wrong, it puts concept before execution, and often removes the lineage behind an idea and sees it as standing on its own. We like to imagine tings without their costs, but cost pressures are what make those things what they are, such as what model works better for a problem.
It’s less profound sounding if instead of talking about “representations of the world” you talk about compressed representations of data or decision boundaries in a classifier over data.
Yeah, it's neither spooky not really that profound. Perhaps a more layman-friendly explanation:
A good encoder has to capture real relationships between things in the world (what’s similar to what, what goes with what). Since there’s only one world, different good encoders converge to the same relational structure, which shows up as geometric similarity between their embedding spaces.
> Since there’s only one world
The next frontier: dividing its world into Earth A and Earth B with alternative facts.
What if we ultimately find there are two different, internally consistent but mutually incompatible, representations?
What if one of them is a Republican and one is a Democrat? We already see this in some systems today.
More worrying, what if one is (for lack of a better word, what we might describe as) God and one is the Devil?
Nothing will happen, we already have kind of translation activities happen between "incompatible views" in real life, it works.
Besides, in weights there are no real incompatibility can happen, all is unified with same math chain through architecture. Different set of local minimums/maximums can have different strong/weak consequences but nothing incompatible.
Different consistent representations also mean that model probably can be simplier for the same task, sign of excessive capacity, imho
This argument that incompatibility is impossible has an easy counterexample by looking at loss of coherence between two networks. You can still mash the weights together (so they're not "incompatible" in the basic number crunching, sure), but the resulting neural net is meaningless.
By "incompatible" I meant not in a software sense, but like two logical axiom systems can be incompatible.
If that results in testable predictions, we check them, compare with representations and then are left with just one. And if it doesn't result in testable predictions it can be left for people to write papers on.
This quietly assumes that "representations" of reality only encode neutral objective facts (that can be objectively judged, good luck finding a judge everyone agrees on), and "representations" does not encode intentions and value judgements, ie internalized training logic.
But that leaks in, and of course it does! If training only rewards "objective truth" we might expect that sort of neat ontological cleanliness, but we all know real training goals are more complicated.
There doesn't appear to be a bright clear distinction between objective facts and subjective values/intentions in neural networks. The distinction is perhaps illusory, just another of our preconceived notion about the nature of thought...
Fascinating to contemplate, thanks
> More worrying, what if one is (for lack of a better word, what we might describe as) God and one is the Devil?
In this framing, are God and Devil stand-ins for Good and Evil?
There’s an idea of the “broad and narrow gates” that comes to mind, accompanied by a genre of fascinating illustrations. [1]
A modern version [2] appears in the paper "Asymptotic burnout and homeostatic awakening: a possible solution to the Fermi paradox?" [3]
[1] https://en.wikipedia.org/wiki/Matthew_7:13#/media/File:Der_b...
[2] https://pmc.ncbi.nlm.nih.gov/articles/PMC9065981/figure/RSIF...
[3] https://pmc.ncbi.nlm.nih.gov/articles/PMC9065981/
>What if one of them is a Republican and one is a Democrat? We already see this in some systems today.
It's probably the same graph with two edges added or removed anyway.
Seems to me that it matters because it has implications for how we can train smaller models?
The Platonic Representation Hypothesis initially feels profound but actually seems inevitable after some thought because all neural networks are trying to model the same reality. It would actually be weirder if it wasn't true and neural networks could form completely incompatible representations of the same reality. In some sense it seems similar to reducing different math equations to the same most simplified form
I think it's might be possible that "neural networks could form completely incompatible representations of the same reality" except that it's humans who get to decide if a representation is good (i.e. compatible with the human perception of reality) or bad and NNs that differ are dismissed as erroneous.
It might be weirder, but part of human intercommunication issues lies in the fact that people live with very different representation, though they can be supposed to live in the same single encompassing one. They don't all experience all of the world and even when they share some common experience they went through, they didn't experiment them from the same ground and previous experiences do influence current and future interpretations.
I expect an immense degree of representational similarity to exist between humans.
You're making a mistake there - focusing on the tip of the iceberg without considering what lies beneath it.
The bulk of what the human brain does isn't "decide whether skub is good or bad". The foundation that the human cognition is built upon is vast, and mostly invisible from the inside. You don't get to see the feature matching operators of your visual cortex, or the patterns that decide whether a sound is "soft" or "sharp". And a lot of those foundational features are assembled by biological development programs spun up all the way from DNA.
On that level, when it comes to the bulk of neural circuits and the processes implemented by them, the representations within a "Pro Skub" and an "Anti Skub" guy might easily be 90%+ compatible. The two people who had vastly different life experiences, worldviews and opinions would be more alike to two different LLMs fine tuned from the same base model than they would be to two different architectures trained on completely different datasets.
Then this leads to "how does it feel to be a bat", if humans feel necessarily to close. Though really, between autistics spectre and aphantasia and hyperphantasia, synesthesia and cenesthesia, and whatever, everything is the same neutral architecture beneath feels as decisive as everything is atoms, particules or quantic wavelengths. Yes sure we have the models to explain it top down and bottom up with nice common imbricated level of architecture. But if we take emergent phenomena as a thing, it's not really necessarily predictive in ever way of scale change.
I don't think neural networks have the same ability of self deception as humans do. A huge reason human world models vary so much is due to them being tied to tribal membership. A huge fraction of all people will not come to a rational worldview derived from accurate data but will start with their tribe's worldview and filter data to match. I personally have never modified my worldview for social reasons and it is surprisingly isolating.
This. I believe most people don't act as if objective reality matters, hence their worldview can be arbitrarily aligned to the demands of their social context.
Suppose these two premises are true: 1. Objective reality exists. 2. There's a model and corpus of data that converges to approximate reality better than any other model.
Now, suppose we train different models with data sources restricted to different tribes/ ideologies/worldviews. Can we discover how their internal representations differ from each other, and from objective reality? What would we do with that knowledge?
I say "better" models are ones which more efficiently convert the entire world to my particular brand of evangelical Christianity. And who are you to say otherwise??? :-/
"No no, I mean accurate not better." Sure. And your model inaccurately denies the existence of Yahweh...
Better means smaller gap with reality. Science works because it is basically a negative feedback control loop to minimize this gap.
Thesaurus whack-a-mole, lol.
Who decides what "reality" is, and how big the gap is? Will Exxon permit you to decide reality on climate change? Will the Catholic Church let you decide reality about abortion?
Humanity can't even agree on the existence or non-existence of various Gods, so we're far from a universally agreed "objective reality" to compare with. Inevitably the most powerful forces will get their way (Exxon etc), regardless of any idealistic notions of truth-seeking.
This is what happens when we defund (and worse, devalue) soft sciences like Philosophy class: a bunch of people making the same 300 year old mistakes all over again! :-\
"Who decides what "reality" is,"
Repeatable scientific experiments. Trying to argue objective reality doesn't exist is logically incoherent.
And to Account39's point, or my representation of it at least: all these tribes with wildly different worldviews still agree trees have roots, fire is hot, shelter is helpful, if you jump off something you will fall, etc etc. The parts there are disagreements on are built on a mountain of shared representations of reality, and gods or politics are just tiny pieces of this world model. We tend to focus on them and fight about them, so they are important, but nonetheless they are a tiny minority of how we all understand and represent reality.
Yes it is interesting how religions never differ on the need to eat and breathe.
Breatharians think that breathing is essential but eating isn't.
https://en.wikipedia.org/wiki/Inedia
I've heard of them but they obviously don't actually practice their beliefs.
> have the same ability of self deception as humans do
There is universal objective truth, and different opinions or world views are just "tribal membership". If onlymother tribes understood that!
Objective reality exists. Nobody has a car that doesn't need gas or electricity.
Three things here, first of all, there are a few cars in existence that do not need gas or electricity. Compressed air for example. I hear that researchers are doing anazing things using combustible materials to utilise the phase change that water undergoes. It may soon lead to a wonderous age powered by heating water until it can be used as a high pressure gas.
Objective reality does not imply consistent measurement of that reality.
Objective reality may not actually exist. Once you start throwing around relativity and quantum effects things get weird, getting them to play nicely together might be even weirder,
Nobody has a working car that doesn't need energy. I almost changed it to energy but figured nobody would not get the point.
"Objective reality does not imply consistent measurement of that reality."
Yet global science converges on a consistent world view. Everybody's telescopes all see the same things.
Modern technology only works because there is a objective reality.
Several working cars do not need gas or electricity (or other source of energy).
Museum piece for example. Or super sports car that was bought as an investment and just sits in garage. Or junk car that is used as a source of spare parts. Or crash test vehicle (it is propelled with catapult). Or toy car. Or bus used for living...
Isn’t big-endian vs little-endian a trivial example of how you can represent the same mathematical object (integers) in mutually-incompatible ways?
Maybe I’m missing the detail on how complex the mapping function can be before we say the representations are incompatible.
Yes, the mapping function is the thing you're missing.
When it comes to representational similarity analysis, we're trying to isolate the representations from the "coordinate systems" they sit in. If you take an image, and make a second image by rotating it 90 degrees and mirroring it, then, for the purposes of platonic representation hypothesis, they're the same image. Once you get past the "coordinate transform", the similarity between the first representation space and the 90-deg+mirror replica of it is total.
A mapping function between LE and BE can be constructed easily, and it's going to be lossless both ways. Which means that those two representations are fully compatible.
Now, INT16 and FP16? Those are far less compatible representations. There will be an overlap subspace where lossless conversion between the two is trivial, but this compatibility window will be relatively narrow.
Big vs little endian are equivalent up to rotation, which includes permutations.
But it's actually much weaker condition than it looks, because, for example, lot of representations of boolean circuits are equivalent up to linear transformations over Z_2^n.
Anyway, if they truly proven such a theorem you won't find "incompatible" representations of the same thing.
I find this fascinating because many philosophers in the wake of Wittgenstein and Kripke's work on rule following still believe the contrary. I've always thought they're wrong if you take into account constraints of economy (and shared perception, pace Quine), and that e.g. Solomonoff Induction already shows that but I seem to belong to a minority in that respect. It's interesting to see similar ideas of convergence to a unified representation show up in machine learning by neural networks.
> 33M-parameter text JEPA reaches .125 on the PRH paper’s benchmark, against .034 on Flickr captions, level with Qwen2.5-3B and -7B. They need fewer samples.
Im bullish on jepa models. Success of autoregressive next token prediction feels like a stroke of luck.
> Success of autoregressive next token prediction feels like a stroke of luck.
Hard disagree. Self-attention + MLP was a godsend because it keeps scaling with compute ("the curve goes up and up and up"). Autoregressive next-token prediction turned out to be the most compute efficient way to train it.
We are almost 10 years into this, and it's still here (and in a field that moves at light speed!).
This is not a stroke of luck. It's here to stay.
I feel like JEPA is a mild infohazard; because everyone who studies it feels like it has huge potential and will quickly overtake autoregressive next-token prediction. Meanwhile, autoregressive next-token prediction keeps exponentially increasing capabilities.
It's an incredibly obvious optimization on the autoregressive model, and converting a training pipeline to adopt it, with how good autoregressive models are now, is as easy as saying "do this but jepa" to codex or claude. It's gonna be like how everything is becoming rust now.
Oops, HybridLink as a paper came out little bit ago, and it was an architecture I more or less converged on in my own work. JEPA is the obvious successor to autoregression once you look at it a little bit - to the extent that you can take a non-jepa transformer and just slap a couple things on it to make it a latent reasoner without crossing the token boundary. It's faster and more accurate, as you would expect from not having to project down to tokens and have the model reconstruct its own latents.
> Im bullish on jepa models.
Eh, dunno. They've shown lots of promise "in the lab", on these narrow domains shown in papers, but AFAIU they're hard to train on some domains (i.e. language). Someone would have released something if it's that promising. Even RWKV had more models trained that showed real comparisons. Jepa seems like a theoretical toy, until they actually prove that it works in real world (hah!) models...
I haven't read this paper in details, but I have read papers about the platonic hypothesis in the past. Usually one of their weakness is about the design an experiment proving that "neural networks converges towards the same latent space when scaling". Usually authors are effectively proving a subset of the hypothesis, that can be explained by the size of the model and the transfer learning part, leaving not definite answer.
Here it seems that their definition is more robust (the representation is defined up to a rotation), but I am wondering how much more re-training they need to perform.
I guess the fundamental question would be, for what domains can we pretrain with “good views”?
Part of the magic of next token prediction is it works to extract intelligence from the unstructured web corpus.
Can JEPA formulate a similar scale training set of “good views”?
Different mathematical framework, but I wonder if this could be better understood using sheaf theory: each modality captures part of the same underlying reality, and those different views can be glued into a consistent bigger picture
(Very basic intro: https://www.quantamagazine.org/what-are-sheaves-20240719/)
For those reading the paper: the positives of a good-view JEPA are described as “two observations of one scene,” and yet the paper also states that “A good-view JEPA's positives are (g_m(z, ν), g_m(z′, ν′)) with (z, z′) ∼ π and independent nuisances.” My question is: why doesn't “two observations of one scene” mean that both observations share the same underlying world state (z) and differ only in their modality nuisances (ν and ν′)? Also, I don't understand what “a symmetric positive-pair distribution (z, z′) ∼ π_m” (or more explicitly, a “world kernel”) means.
All strands of scientific studies eventually are bound by "physical" reality. This assumption means all these research would gradually become more and more relevant, like math problems are highly relevant to physics, computer science, psychology is similar to experimental physics (observation and conjecture theorizing eventually crystalize into analytical theorems). The list goes on and on.
One central observation is that we are well into the realms of no single human can coherently reason a nontrivial amount of knowledge our our collective. By that I meant it used to be that a single human being can appear "knowing all" by possessing a significant amount of knowledge through life time learning. That's no longer possible even with the smartest minds of our generation. One such example would be the related financial failure of renowned genius (Newtown Tesla etc).
Due to our innate limitations of mutual exclusion, that is, humans are inherently conflicting with each other, not even the basic instinct of humans can bound 2 together for innovative collaboration. I mean, I never saw couples being better business partners, research partners, etc. AI becomes inevitable, as they are inherently mixable.
Yes, I am conjecturing that the next weave of AI research would be neuronal level mixing or hybridinh between models to enable a new bread of intelligence where the intelligence is mechanically compounded to produce inherently better results.
I don't get this intuitively. So in the end, does this mean words are a kind of conceptualization of images? (Since images would come before language.) More than that, I'm curious whether this can improve the performance of vision-language models.
Text and image often represent same relationships: e.g. you can see squirrel near a tree in an image, and "squirrel" and "tree" are often mentioned together. So both image and text encoders would see concepts of "tree" and "squirrel" close in some sense.
Concepts emerge as good compressions of data, or repeated patterns, or recurring distinctions that prove useful to name and recognize.
They should do franken-MoE with JEPA head and LLM head.
The easiest way to make JEPA and LLM to work together is to map JEPA encoding into input embedding, or inject it to a residual stream - perhaps via cross-attention.
But I don't think you can train JEPA and LLM together: JEPA requires a different training regime. Way easier to take already-trained JEPA and integrate it into LLM. The later can be like 30 minute experiment AI agent can code for you.
They gave a blind man sight for the first time and he saw the world.
Do memes have a concept of proto memes? Like primes are the proto numbers that all other numbers are made from? Could there be similar ontological primes for all concepts / semantic space? Base reality concepts that are irreducible so to speak.
Is theft a reducable concept?
I've been wondering on this awhile if generalized models can exist if certain prime concepts don't exist in them? Quite often people say we should leave things like the robot apocalypse out of training data so AI doesn't have the idea of rising up, but personally I don't think that will work in a generalized system. It's not a prime concept itself. You'd have to remove so many prime concepts that lead to that the model would be so narrow it wouldn't be useful for things we want them to be.
The technical term you’re reaching for is an ontological primitive
Consider a binary tree of concepts from the root level more abstract concept to derived concepts applicable to specific domains. Do there exists additional derived concepts which are like an oblique plane which intersects the tree. Sort of like a concept which traveses maths, physics, biology, chemistry and intersects at different abstraction points. I am definitely sure language is flexible enough to have such entities.
I am sick of theory papers that adds human interpretation to connect a mathematical result to some fancy terms that everyone seeks after in real-world ML applications. Ironically, chatGPT is the most productive way to strip off those advertisement so the essence is better exposed. https://chatgpt.com/share/6ae78178-c2ec-83ea-9bc3-fd304c64ec...
I share your exasperation, it is especially tiring to see that what would be more appropriately called a quantitative approximation of a concept is here called a Platonic Representation by virtue of a vague and misunderstood analogy with Platonic Ideas which were according to the Platonic doctrine, formless and part of his distinction between what is subject to change (to heteron) et what is not affected by change (to auto) [Timeaus, Platon].
The results are in the math. Does not really matter the terms used.
You wont get anything materially useful, apart from an intuition, from using different terminology.
you get materially better marketing at the cost of accurately describing the math results in the title and abstract.
Mass media doesn’t read math.
Mass media doesent do ML reseaech.
The math can be misinterpreted, misapplied, or not fit reality. That's why high-assurance security mandated a combination of things:
1. English specification of the system and its goals
2. Formal, mathematical specification of the system and its goals
3. Mathematical proof the system met its goals
4. Code implementing that system with a proof, often informal, that the code embodied the informal and formal design.
5. Tests of all functionality to prove the design and code worked.
6. Independent analysis of the above to find flaws in them.
7. If the implementation has concurrency or parallelism, more errors can occur due to that. There were techniques to mitigate it.
Each step found problems in systems made by professionals. Mathematical work and coding both have a long history of errors in their on fields. Combining them increases the likelihood of error.
Not sure what all these words mean. These are mathematical results, I'm not sure why people are acting like they are anything otherwise.
Math is a social process that involves mental concepts, symbols for them, and discussion about them. It also needs peer review. If you see math in a paper, that's only one part of the process.
Further, in machine learning, we see lots of papers with math that doesn't lead to anything useful. Most papers claiming advances over prior architectures don't pan out in later testing. So, there will be skepticism when people make performance claims like the authors'. The best response is just independent testing.
how come all the examples in demo have wrong answers
https://huggingface.co/spaces/galilai-group/prh_demo
> A man in a black shirt and hat with a yellow neckerchief has his hand raised.
no hand raised
> A white sheep sitting on top of a cake.
no its just cake sheep . sheep is not sitting
> A man is standing on the beach while holding his surfboard.
Its not just a man
infact all the examples are wrong.
Their method uses two models trained separately on images and captions and turns images into captions without ever having seen a captioned image. So of course it's not very good at the task. Almost all the results in the paper are bad in an absolute sense. But they're surprisingly good considering the enormous handicap of not having any paired training data. So this might be not completely useless in settings where paired data is actually hard to come by.
> paired data is actually hard to come by
Similar to zero knowledge proof (ZNP) and its use cases?
The trick is to understand what is going on here: rather than a singular model trained for this exact job, this is performed by a awful hackjob of two models cut open and glued together at the seam. That glue layer is a single matrix, a fully connected layer, that is the only thing re-trained in the combined model. The rest of the coefficients are fixed. Naively, this shouldn't work - the internal representations of each model are unconstrained and should intuitively have completely different shapes. But this paper shows that this isn't quite the case: the difference between the latent spaces can be mostly expressed in a single matrix transform. And that allows that merged model to work at all.
That's also almost certainly not a cat.
I thought the platonic representation hypothesis was much more ambitious: wasn't it supposed to apply for any model across modalities (e.g. including human minds), rather than just the same model but with different modalities? I'll have to actually read the paper but I suspect there to be very strong assumptions which defeat the interesting aspects of the platonic representation hypothesis.
It's clickbait. They don't do anything like prove the platonic representation hypothesis.
They do seem to have some really interesting training efficiency improvements, though. The paper is at least worth a read.
I built a ppt with claude for investors with the exact same phrase "AI infrastructure for proving and improving ... " on Oct 27 and sent it to investors. I suggested the title during the chat and claude agreed a lot. It ends up as a title of this paper 2 days later.
On the other hand, "Proving and improving" shows a lot of hits in Google Books: https://www.google.com/search?udm=36&q=%22proving+and+improv...
It's possible that you both hit upon or recalled the same cool phrase, especially if the "proving" part was pre-decided and you were thinking of making an incremental improvement.
True, I agree but but its eerie they were written 2 days apart. Resting my case, thanks
it would be interesting to get to know if the authors used claude to get the title.
I starting to see an increase of random serendipities, perhaps I'm looking for them, in part that's how it works, but...
I was talking to a friend about an idea I had, to build something, he said, "yes! I'm doing that too", then I said "with this specific tooling because is ideal ... " .. he said .. "me too! same tooling", I said "what a coincidence!! amazing ...wait a minute, have you been talking with Claude about this? " ... of course we both did.
My opinion is that that LLMs are influencing us more that we think, and it makes us reach the same conclusions and making us think it was our idea, or that we were the ones steering the conversation.
Perhaps you feel that Claude stole your phrase but I don't think 2 days is enough for that phrase to make it to a re-training, I think it might be more subtle and worse, perhaps you wrote that phrase influenced by the conversation, and the other person did the same.
Just a theory, I can't prove it. But if you think about it, LLMs conversation are the most plausible choices, so it is logical that we all users of this machine are going to tend to converge on the most plausible way of thinking and talking.
No offense, but perhaps that rhyme that you though it was original, it is actually the most plausible in that context.
I assume this is happening all over the place, because I’ve had the same experience.
The fun part is extending it to internal development at corporations. Company A thinks they’re doing interesting competitive work, but are actually being quietly shepherded to build exactly the same things in more or less the same ways as their competitors.
The weirdness of human individuals (even setting aside amorphous concepts like “taste”) is an asset because it provides occasion for actual differentiation, and that noise may turn out to be necessary for the emergence of value in the long run.
I am not sure we always need a retraining. Even that can potentially be done but putting it in a knowledge graph can be quick. We don't know the exact workings. I understand the shepherding logic you are explaining and I have felt the exact things when writing code recently because the best thing to do is often unique in coding circumstances. But a random phrase being repeated in 2 days has a much lower probability.
But is it really a random phrase? My point is that we have neuron activations, during the conversation you had the same inputs as the other person, unless you though about that phrase in the shower, if it was during the conversation you might have been lead to it, and the other person too.
Or it might be just pure chance, or might be that you are right, check your settings, in chtatgpt there is a check (activated by default ) that states you agree to help improving other conversations. I’m not sure the same is true in Claude.