🔍 Read the full analysis: Weight Space Programming: A New Way To Explore How AI Works on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
MIT CSAIL researchers say GPT-6 Astra could identify behaviors in small neural networks and write or edit their weights without a training loop. A follow-up report describes mixed results in three additional trials, including a narrow benchmark improvement and a legal-task score that did not improve. The method’s reliability and safety effects remain open questions.
MIT CSAIL researchers reported that OpenAI’s GPT-6 Astra could identify, create and edit small neural networks by working directly with their weights, without using a training loop. A November 6 account by Thorsten Meyer, updated November 9 with results for 10 published legal tasks, describes separate tests with mixed outcomes, suggesting the approach may help inspect or repair some models but is not yet a general-purpose substitute for training.
In experiments described in a November 2 MIT CSAIL blog post, researchers Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab gave GPT-6 Astra network weights as text and asked it to reason about them. The researchers said the model identified the games represented by four small networks, each with 17,253 weights. They also reported that it found copying heads—components that repeat patterns from earlier text—in three small language models.
The team also tested whether the AI could write networks from scratch. According to the MIT account, Astra generated weights for networks that played three arcade-style games and outperformed comparison networks trained for 100 million steps. The researchers said all experiments barred training and that prompts, responses and evaluations were published on GitHub. The source account does not provide enough detail to independently assess every evaluation or reproduce the results here.
Meyer’s report describes three additional trials using two AI systems. In one, an edit to Qwen3.5-0.8B removed one direction of a looping behavior by changing 0.016% of its weights; the reported MATH-500 score rose from 7.9% to 48.1%. In a separate local-model test, Claude’s edit let the model answer all 10 published legal tasks rather than four, but its legal score remained zero. Another AI review identified faults missed by green tests, while a one-matrix edit raised a narrow JSON test result from 49 to 70 of 120. The report says a third experiment produced no useful improvement.
Infographic 01 · What weight space programming showed
Read, write, edit: what GPT-6 Astra did with raw weights
A blog post with complete public logs, not a peer-reviewed paper. Small networks, one research group, one run per setting.
MATH-500 score of the small open model Qwen3.5-0.8B after GPT-6 Astra removed one loop direction from 0.016% of its weights. Before: 7.9%. A Hugging Face re-run of the edited model: 51.0%.
Three things an AI did with raw weights
No training loop, no gradient descent
- Read: named the game behind 4 of 4 small game networks (17,253 weights each) and found every copying head in three small language models with their first and last layers removed.
- Wrote: arcade-game networks from all-zero weights that beat networks trained for 100 million steps in all three games. Breakout: 517.4 against 25.3 for 500 million steps.
- Fixed: a car network that never backed up: 0 successes in 256 episodes before, 1,024 of 1,024 after, with 3.5% of the weights changed.
Fixes what is broken, adds nothing new
Gemma 3 270M rarely loops (8.5% of answers hit the limit). No edit moved its MATH-500 score by more than about a point.
Some things stay invisible
Honest model or yes-man? From the weights alone: not found in 4 of 4 attempts. With a view of the running model: found in 3 of 3 pairs.
Sources: MIT CSAIL blog post, 2 Nov 2026 · logs on GitHub · Hugging Face re-run.
Powered by Thorsten Meyer AI
A New Route to Model Repair
If the results hold up, direct weight editing could offer researchers another way to investigate open models: rather than retraining a network, an AI might inspect selected parameters, identify a behavior and make a limited change. That could reduce the time or computational work involved in some debugging and auditing tasks. The reported tests, however, cover small models and specific evaluations; they do not establish that the method works reliably on large commercial systems or across tasks.
The same capability has a safety dimension. Meyer’s account warns that a weight edit could remove safety training as well as repair unwanted behavior. That is a stated risk, not evidence that the experiments caused such a change. For organizations using open models, the report makes weight provenance and change review relevant: a modified model may behave differently even when only a small fraction of its parameters changed.
Popular right now
KitchenAid Artisan 5-Quart Tilt-Head Stand Mixer, Almond Cream
Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB‑C Charging, Up to 2X More Active Noise Cancelling, Transparency Mode, Adaptive
Ninja Foodi 10-Qt DualZone Smart XL Air Fryer, 6-in-1, DZ550
As an affiliate, we earn on qualifying purchases.
From Training to Direct Editing
Neural-network weights are numerical parameters that shape how signals move through a model. Conventional training adjusts those parameters repeatedly, using an optimization process such as gradient descent to reduce errors. The resulting behavior is distributed across many numbers, which can make a model difficult to interpret by looking at its weights alone.
Researchers in mechanistic interpretability have developed techniques for finding structures and behaviors within networks. Meyer’s article cites 2021 work by Anthropic researchers on detecting some algorithms directly from weights. The MIT team’s proposal is to use a general-purpose AI model to reason about those parameters and edit them deliberately. The blog post describes the ambition as neural-network weights being “natively optimized by LLMs that deliberately reason about their weights.”
The two kinds of work should be distinguished: MIT’s reported experiments are the research development, while Meyer’s three trials are additional tests described in a separate account. The latter do not by themselves establish that the method is broadly effective, and the legal-task result illustrates why answering more prompts does not necessarily mean a task-specific score has improved.
“Our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.”
— MIT CSAIL researchers Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, as quoted in their blog post
Limits of the Current Tests
The available accounts do not establish how well direct weight reasoning transfers to larger networks, different model families or real-world deployment. The MIT experiments were conducted with GPT-6 Astra, and the reported findings depend on particular small networks and evaluations. Public prompts and runs support scrutiny, but the supplied material does not establish independent replication of the results.
It is also unclear how often an edit that improves one measure would harm another behavior, or how reliably reviewers can detect such side effects. In Meyer’s legal test, the model answered more tasks after editing but retained a score of zero. That result leaves open what “answering” meant in the evaluation and whether the edit produced useful legal reasoning. The source describes a safety concern about removing safeguards, but gives no measured safety evaluation of that risk.
Replication and Safety Checks
The MIT team has published its prompts, model responses and evaluations on GitHub, giving other researchers material to inspect and test. The next meaningful evidence will be independent replication, broader evaluations across model sizes and tasks, and checks for unintended changes after editing. The source material does not identify a scheduled follow-up paper or release date.
For now, the work is best read as an early research report, not a ready-made method for modifying deployed systems. Researchers and organizations considering weight edits will need to document the source of each model, compare behavior before and after changes, and test both intended improvements and possible losses in safety or capability.
Key Questions
What is weight space programming?
Weight space programming is the idea of asking a general-purpose AI model to inspect or change another neural network’s numerical weights directly, rather than training that network through repeated optimization.
What did the MIT researchers report?
The MIT CSAIL team said GPT-6 Astra identified the purpose of four small game-playing networks, detected copying heads in three small language models, and generated weights for networks that beat comparison networks trained for 100 million steps in three games.
Did the follow-up tests show consistent improvements?
No. Meyer’s report describes mixed results: one edit improved a MATH-500 score and another raised a narrow JSON test result, but the legal score remained zero and one experiment produced no useful improvement.
Could editing weights affect model safety?
The source report warns that direct edits could potentially remove safety training as well as repair unwanted behavior. It does not report a measured test showing that safeguards were removed in these experiments.
Is this ready for general use?
The available evidence concerns specific experiments on small networks and does not show that the approach is reliable across large models or deployment settings. Independent replication and broader safety testing are still needed.
Source: ThorstenMeyerAI.com
