Andrew Saxe @SaxeLab

Why do humans and LLMs both show working-memory limitations? They may face the same computational trade-off: shared representations enable efficient learning and generalization, but also create interference. I’ll be presenting this work at COLM Oct 6 (today!), Poster #32, 4:30PM

This is outrageous framing. Quite frankly, given the current tensions around AI and mathematics, this kind of announcement does not help. After the recent Buckmaster/OpenAI dispute over scientific credit and provenance, the last thing we need is more promotional PR language that blurs the history of a research problem. Meta presents its work by asking: “Can AI contribute when a problem is genuinely open and without an existing solution path?” Seriously?! For the ellipsoid-fitting example, this is misleading and insulting. There was an established research programme, substantial rigorous progress, and by the time of their announcement, publicly available proofs of the exact threshold. Let us be specific: - In 2023, Antoine Maillard and Dmitriy Kunisky gave a statistical-physics derivation of the conjectured transition at n/d² = 1/4, using the replica method. Their analysis also predicted the geometry of the solutions and suggested directions for rigorous proofs. This was not a proof, but it was a detailed and successful theoretical picture, not an absence of a path. arxiv.org/abs/2310.01169 - Afonso Bandeira and Antoine Maillard then rigorously established the sharp threshold for approximate fitting, using methods inspired by the statistical physics of disordered systems, using actually spin glass mathematical technics. Their preprint appeared in 2023 and was published in the Electronic Journal of Probability in 2025. Their Gaussian-equivalence framework provided a foundation for further work (including many of mine!). At this point, the "exact" fitting and removing the spectral restriction were the outstanding gaps, but these are technical problems, it was not about not discovering the threshold from scratch. arxiv.org/abs/2310.05787 - On 10 August 2026, Theodor Misiakiewicz and Garrett Wen posted a proof of the exact Gaussian threshold, explicitly closing those two gaps. arxiv.org/abs/2608.10184 - On 12 August, Sofia de la Cerda, Aaron Potechin, Madhur Tulsiani and Jeff Xu posted another proof, establishing both sides of the transition up to a vanishing multiplicative factor. arxiv.org/abs/2608.12415 - On 27 August, Frederic Koehler and Youngtak Sohn posted a broader universality theorem that includes the Gaussian threshold. arxiv.org/abs/2608.27372 These were not researchers merely “on the verge” of solving the problem. THE PROOFS WERE ALREADY PUBLIC. Meta's announcement comes in October, and Section 3 of their paper begins: “We use the groundbreaking approximate-fitting theorem of Bandeira and Maillard…” Dear META: Those are your words. This is not simply a reference in the bibliography: your feasibility proof explicitly starts from their theorem. How is that consistent with advertising this example through the suggestion that there was “no existing solution path”? Yes, your social post mentions independent solutions, and your paper and blog identify the August results and describe your work as independently developed. That does not answer the objection to the framing. A generic acknowledgement of parallel work does not make clear that the exact threshold already had publicly available proofs before this announcement. An independently developed AI-assisted proof, released after other proofs, should be presented as exactly that. Readers should not have to reconstruct the chronology to understand what is, and is not, new. A new proof can contain valuable ideas. But a new proof and a newly solved problem are not the same claim. Nor does a problem being open when a project begins mean it is still open when the marketing campaign launches. You say the aim is to “empower researchers”? Nothing more “empowering” than seeing years of collective research turned into the backdrop for a cheap marketing stunt. Correct the announcement! Give the scientific history the same prominence as the product. Give Antoine, Afonso, Theo and all the other researchers involved the public credit they deserve.

AI at Meta @AIatMeta

2 weeks ago

1️⃣ Probability: Mathematicians worked with Muse Spark to answer a question about fitting random points onto the surface of a stretched sphere. For the setting studied, they proved a sharp cutoff between when an exact fit is likely and when it is unlikely. Read the paper:

11 7 157 97K 48

The difference between industry PR and rigorous academic discussion. Worth a read.

Krzakala Florent @KrzakalaF

2 weeks ago

This is outrageous framing. Quite frankly, given the current tensions around AI and mathematics, this kind of announcement does not help. After the recent Buckmaster/OpenAI dispute over scientific credit and provenance, the last thing we need is more promotional PR language that

25 122 704 81K 233

Are brains and artificial neural networks converging onto universal representations? There is a seductive idea making the rounds in NeuroAI / ML: train systems well enough, and they'll converge on the same representation of reality (i.e. unique world model). We have Thoughts™

New preprint: brain alignment in DNNs is almost complete within the first 3% of training, while classification performance is still low. The features involved in classification are largely independent of those that predict the brain (@_niklasmueller , @IrisGroen , @marcelge )

Join us at EPFL as a professor in Mathematics of Learning: epfl.ch/about/working/…

Tomorrow, I head to beautiful Gothenburg, Sweden, to lecture at Analytical Connectionism (analytical-connectionism.net/school/2026/). In a shocking turn of events, I will talk about controlled rearing -- a topic I have never before presented!

Why do all neural models exhibit phenomena of sudden learning (and scaling laws)? In our most recent preprint, we showed that this is due to permutation symmetry. At a small initialization, permutation symmetry reduces the training dynamics to depend only on the architecture-specific structure matrix A. arxiv.org/abs/2608.13335

Natural data has a hidden hierarchy - and deep architectures can learn it from remarkably few examples. That one idea explains a lot: why these machines get creative with limited data, why neural scaling laws appear, and why learning from your own latents beats learning from tokens. My full conversation with @ecsquendor @MLStreetTalk is now out - links below.

New preprint from the lab! Behavioral and neural mechanisms for stochastic choices in mixed-strategy games: doi.org/10.64898/2026.… We use game theoretic tasks, cross species comparisons, modelling strategies, and mesoscale imaging to probe how we can escape predictability!

Version 2 of Theory of Contravariance is out! arxiv.org/pdf/2607.08561 New material on contravariance for Transformers, and the theory of Representational Similarity Analysis (RSA) and centered kernal analysis (CKA). For transformers: it turns out that they have privileged axes, just like convnets, if you look in the right place (MLP layers and attention heads.) The identification of privileged heads is a potentially key result for emergence of interpretable stucture in LLMs. @meenakshik93 For RSA: it turns out that you can decompose RSMs into unique task-relevant "core geometry" and a task-irrelevant symmetry-generated term. Weak-strong equivalence holds for the core geometry and by projecting onto privileged axes you can filter out the task-irrelevant part so it doesn't interfere. This builds on work from Marvin Theiss @sciencelukas @saxelab @ermgrant The Principle Investigator will do a deep dive on both topics in the coming days! danyamins.substack.com

Presenting your work with friends is always nicer! Make sure to checkout Neuralplayground published as part of @CogCompNeuro proceedings! openreview.net/forum?id=kHLpX…

The amount of basic research done in industry is nowhere near the amount done in universities. At various times, there have been industry labs that had fundamental research activities and have made important scientific contributions. Examples in information technology include Bell Labs, IBM Research, Xerox PARC, GE, Phillips, NEC, and several other. That disappeared in the 1990s. Microsoft Research picked up the torch in the 2000s, followed (to some extent) by Google and then Meta (for about a decade until recently). But their innovations almost always built on top of academic work, and certainly profited from the whole research ecosystem.

This is the very first project I started working on during my PhD with the amazing PPSleep team, and I’m so excited to finally see it published! 🎉 🧠Replay of procedural memory is independent of the hippocampus nature.com/articles/s4159…

New paper w/ @SaxeLab & Nishil Patel! 🧵 Does RL post-training teach models anything new, or just amplify skills already in the base model? We built a fully auditable testbed to settle it — and caught RL composing new strategies in the act.

A question on synthetic data generation: If we want a language model to solve k-step arithmetic problems (such as a+b*c-d=?), with operands from 1 to 100, which training distribution should we use? A. Uniform distribution: Sample these k operands uniformly from 1 to 100 B. Power law: randomly shuffle 1-100 and impose an artificial power law. Sample these k operands according to this power law. ⚡Our ICML 2026 (spotlight) paper shows: Option B is better! Surprisingly, the same idea extends far beyond this simple example to many reasoning tasks that require implicit composition of multiple atomic skills, including multi-hop QAs and synthetic GSM problems. 📄Paper: arxiv.org/abs/2604.22951 📝Blog: zixuan-wang-dlt.github.io/posts/2026/06/…

Pretraining + fine-tuning powers modern ML, but we lack a theoretical understanding of how pretraining actually shapes downstream learning. Enter our new @icmlconf paper: “A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning” 📅 July 9th, Poster #4502 Session 8! 🧵

Interested in any of our works? Come and say hi @icmlconf! Unfortunately I won't be around this year but my collaborators will answer all your questions :)

Cognition, Adaptation and Learning (CAandL) Lab @caandl_lab

3 months ago

We are excited to share that CAandL Lab will feature in 3 papers at #ICML2026 this week. Stop by and say hi if you are interested in any of these! @devonjarvi5, @stefsmlab, @geraudnt with @kleinric, @BenjaminRosman, Damien Harvey, Branden Ingram, and Steven James

0 6 10 2K 0

Is Muon as good as they say? We looked beyond training speed and found a hidden cost: Muon loses the simplicity bias of older optimizers like gradient descent — and this matters for generalization.

Very proud to be leading the SOFAIR Lab ucl.ac.uk/engineering/so… with fantastic colleagues from UCL, Edinburgh, Oxford and Cambridge.

Kanishka Narayan MP @KanishkaNarayan

4 months ago

🚨 2 major new AI labs in the UK 🚨 3 months ago, we launched a £40m call for a fundamental AI lab in the UK. Given the exceptional bids we received, we have doubled down: 2 new AI labs, £60m seed funding. ▪️@bold_lab_ai: led by @j_foerst, with an exceptional team of

7 30 149 25K 34