🔍 Read the full analysis: Can Neural Network Weights Help AI Write And Repair Models? on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
Four MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could infer what small neural networks did from their weights, repair some failures and write new networks. The results come from a blog post and logged experiments, not a peer-reviewed paper; the tests were limited, and Astra could not solve every task.
Four MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could read the numerical weights of small neural networks, identify some of their functions, repair certain failures and write new networks. Their November 2 blog post calls the approach “weight-space programming”; it describes promising demonstrations, but the work is not a peer-reviewed paper and its tests remain limited.
In conventional machine learning, training software adjusts a network’s weights—the numerical connections between artificial neurons—through repeated optimization. The researchers instead asked Astra to reason about those values directly. In some tests it received weights as plain text and answered in one response; in others, researcher-run sandbox code evaluated programs Astra wrote. The team says the model had no built-in tools such as shell access, web search or file editing, and that it reviewed 1,125 turns in the released logs.
In reading tests, Astra identified the game represented by four small game-playing networks and identified the shape generated by three small image generators. It also found the researchers’ identified “copying heads,” a mechanism in transformer models, in each of three tiny language models without running those models. These are results reported by the research team, not independent evaluations of a broad set of systems.
The repair demonstrations included a car-control network that had recorded zero successes in 256 episodes before Astra edited it. After a single response and a change to 3.5% of its weights, it succeeded in 1,024 of 1,024 episodes, according to the post. In a separate test, changing 0.016% of Qwen3.5-0.8B’s weights was associated with a MATH-500 score increase from 7.9% to 48.1%. The source says a Hugging Face engineer rebuilt that edited model from the logs and obtained 51.0% on MATH-500.
Astra also wrote networks from all-zero weights, including a 1.5-million-weight Go network. The post says that, across 50 test runs, the Go model beat a same-size comparison network trained on about 20.5 million games. For smaller games, the team reports that Astra-written models beat comparison networks trained for 100 million steps. Those findings depend on the described test settings and do not establish that the method can replace standard training generally.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A Different Route to Model Repair
If the results hold up in broader testing, direct reasoning about weights could offer researchers a way to inspect or patch some small models without running a conventional training process. That could be useful for targeted debugging, auditing model behavior, or experimenting with open models whose weights are available. The Qwen result is especially striking as a reported benchmark change, but it is one model and one task in a single research group’s account.
The approach also shifts attention to the provenance and integrity of model weights. If an AI system can make meaningful edits to another network’s parameters, developers may need stronger checks on where weights come from and what changed between model versions. The experiments do not show that users can safely or reliably repair production systems this way, and the source says the work does not change anything for users today.
Popular right now
De'Longhi Rivelia Fully Automatic Espresso Machine with Milk Frother, White
Breville BES870XL Barista Express Espresso Machine, Brushed Stainless Steel
De'Longhi Dinamica Plus Fully Automatic Espresso Machine
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, computer scientist Andrej Karpathy described neural networks as “Software 2.0”: programs whose behavior is encoded in learned weights rather than written line by line. The MIT researchers’ project takes up the question of whether a language model can treat those parameters as something to inspect and edit. Their blog post describes the weights as being handled without an optimizer or gradient descent in the relevant demonstrations.
The researchers—Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab—used OpenAI’s GPT-6 Astra through the Codex command-line tool, according to the released logs described in the source. The work appeared as a blog post on November 2, 2026, rather than as a peer-reviewed publication or an arXiv paper. That status matters: public logs make the experiments inspectable, but they do not substitute for replication and review.
““not through an optimizer, and in particular not with gradient descent””
— MIT CSAIL research team, in its November 2 blog post
Limits of the Reported Tests
The available evidence is an unreviewed blog post with public logs, not a peer-reviewed study. The researchers report one run per setting, small networks, one model family per task and a single research group. Those limits leave open how often the method would work if repeated, whether results would carry over to larger or more varied models, and how sensitive the findings are to test design.
The post also records failures. Astra could not distinguish an honest model from a sycophantic one by inspecting weights alone. Its edits did not improve maths performance in Gemma 3 270M, and the researchers say editing could not supply an ability the model did not already have. The supplied source ends partway through its discussion of how easily some weight patterns might be interpreted, so it does not provide the full account of that limitation. Broader reliability and practical safety remain unestablished.
Replication and Broader Evaluation
The next test is whether independent researchers can reproduce the logged results and whether the method works across more models, tasks and repeated runs. The source describes no scheduled peer-reviewed publication or follow-up milestone. For now, the public logs and the reported Qwen re-evaluation offer material for scrutiny, but they do not settle how dependable weight-space programming is.
Developers and model users should treat the demonstrations as early research, not a ready-made repair tool. Further evaluations would need to establish when direct weight edits improve performance, when they cause regressions, and how to verify changes before deployment. The source reports no immediate change to products or user-facing systems.
Key Questions
What is weight-space programming?
It is the researchers’ term for using a language model to inspect or change a neural network’s weights directly, rather than relying on a conventional optimizer to update them during training.
What did GPT-6 Astra reportedly do?
In the MIT team’s tests, Astra inferred functions of several small networks, edited a failing car-control model, changed weights in a small language model associated with a higher MATH-500 score, and wrote small networks from zero weights.
Has the work been peer-reviewed?
No. The source describes it as a blog post with public logs, not a peer-reviewed paper or an arXiv preprint.
Does this mean AI can repair any model?
No. The reported tests were limited, and the researchers describe failures. They say edits did not improve maths performance in Gemma 3 270M and could not add an ability the model lacked.
Can users apply this to their models now?
The source reports no user-facing change or ready-to-use repair tool. Independent replication and broader testing are needed before the results establish practical reliability.
Source: ThorstenMeyerAI.com
