A study published on arXiv on October 29 by Yann LeCun and Randall Balestriero argues that the size of a language model may not be the decisive factor in how well it develops an internal representation of the world. Their experiments suggest that a 109-million-parameter text-based JEPA can align more closely with image encoders than language models with up to 72 billion parameters.
The paper, titled “Proving and Improving the Platonic Representation Hypothesis with JEPAs,” addresses an idea first observed by MIT researchers in 2024. That hypothesis proposed that models trained independently on text and images develop increasingly similar internal representations as they grow. LeCun and Balestriero claim to turn that observation into a theorem for a specific family of architectures: Joint-Embedding Predictive Architectures, or JEPAs.
According to the researchers, two JEPAs trained on different modalities can learn the same representation, subject to a rotation, when they receive suitable views of the same scene. Examples include an image and its caption. The paper contrasts this approach with next-token prediction, the method used by large language models, where two pieces of the same document do not necessarily describe the same subject.
The reported experiments produced a notable comparison. The 109-million-parameter text JEPA aligned more closely with the DINOv2 and CLIP image encoders than any of the language models tested. On DINOv2, it achieved an alignment score of 0.184, compared with 0.154 for Qwen2.5-72B, according to the source report. The authors also say that the JEPA required 42 times fewer training examples to reach the same alignment as a next-word prediction model.
The experiments covered datasets ranging from 145,000 captions to 940 million images. These results support LeCun’s long-standing view that conventional large language models are unlikely, on their own, to produce an artificial intelligence capable of understanding the physical world. His startup, AMI Labs, has built its research direction around this broader approach.
The findings should nevertheless be treated cautiously. The paper has not yet undergone peer review, and no independent team has reproduced the comparison. Its central score measures alignment with a vision model; performing better on that metric does not necessarily mean that a system will summarize documents or write code more effectively.
The study’s broader argument is that the quality of the data and views presented during training may matter more than simply increasing the number of parameters. The small JEPA is not presented as an immediate replacement for systems such as Qwen or GPT, but its results question whether scaling language models alone is the most efficient path toward richer representations of the world.
Sources: Numerama.