Commenters narrow the claim: a known bug fix, not new math knowledge
3 Nov 2 2:45 PM · 5d ago · 2 posts · 8 comments · 3 sources · development 3 of 5
Several technical commenters point out that Qwen already knew math but was looping past the token limit, and that Astra's fix resembles known single-direction ablation techniques from prior interpretability work, rather than teaching new knowledge.
“Qwen already knew math, but it was bugging out and getting into an infinite loop when it tried to solve the problems (85 of its first 96 answers hit the token limit).”
l9oRikiya Takehi Researcher, author of the blog postGPT-6 Astra OpenAI LLM used as the weight-editing modelQwen3.5-0.8B Target model edited by Astra
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What people said 8 voices · verbatim
-
The math example on Qwen was based on 50 batched iterations, so the model was able to evaluate it's changes on downstream tasks. And the fix was essentially to remove a direction in activation space that causes looping instead of an exit, this was already known to be a thing(all of the jailbreak/unleashed models do this) following Arditi et al…
-
I dont feel the air of impressivness here. It seems like of course so. Of course theres geometric relationships in the data we dont understand, of course a mathimatical relationship machine found it.These models are made the a vanishingly small group of people based on how much a frontier lab consumes, of course theres increasing capability in…
-
I think the "taught qwen math" part is not exactly correct though, even if the post makes it sound like that. Qwen already knew math, but it was bugging out and getting into an infinite loop when it tried to solve the problems (85 of its first 96 answers hit the token limit). With numpy and pytorch, Astra found where the loop was happening and…
-
Just logging in here for the record to state that this property of neural networks is one of the most important aspects of their operation. The convergence of this sort of interoperability, tune-ability, and modularity is probably the most important final frontier in all of ML/AI/software.Kudos to this fellow for having the temerity, or perhaps…
-
I don't think this is an emergent capability. I think Astra was trained on neural weights data as an attempt by OpenAI to get to recursive self improvement.Something similar happen in early GPT models which had a lot of chess game in standard notation in their training data which made them very good at chess as long as your prompt had the right…
-
"No human can go edit a brain to become better at math" -- I'd offer this is exactly what "learning math" is.I know I'm being pedantic, and I understand you mean "edit" like "surgically", but it does still feel worth noting.
-
This has to be a good step to recursive self improvement, no? Give a model it's own weights and tell it to make them better at x,y,z, etc...
-
Did they try it with other frontier LLMs? What about other non-trivial out of distribution tasks? So many questions.
All 5 developments of Researcher shows OpenAI's GPT-6 Astra hand-editing Qwen's… →
MastodonHacker NewsNewswiresLobstersXRedditBluesky