MIT researchers improve a small AI model by editing just 0.016% of its weights - Big Tech and the digital economy

In a preliminary study, a more capable model modified a small portion of Qwen3.5-0.8B’s weights, raising its score on MATH-500 from 7.9% to 48.1%. The method, called Weight-Space Programming, showed little benefit on a model that did not have the same reasoning-loop problem.

Researchers at MIT CSAIL have demonstrated a way for one AI model to improve another without retraining it. Their method edits a small fraction of a model’s internal weights to address a specific weakness, offering a possible alternative to the costly process of training a language model again.

The team used GPT-6 Astra to modify Qwen3.5-0.8B, aiming to improve the smaller model’s mathematical reasoning. After the edits, Qwen’s score on MATH-500 rose from 7.9% to 48.1%. Astra changed just 0.016% of Qwen’s weights, according to the report.

The researchers call the approach Weight-Space Programming. Rather than adding new capabilities through training, the method targets and adjusts parts of a model associated with reasoning loops that can hinder its performance.

The findings are preliminary and have not been peer-reviewed. The method also appears to depend on a model having a problem that can be corrected this way: when tested on Google’s Gemma 3 270M, which reportedly experiences few reasoning loops, the improvement was negligible.

The work points to a possible way to improve smaller, less expensive models, but it does not replace training as a general solution. The researchers also suggest that models might eventually apply such edits to themselves, though that possibility remains a future prospect.

Sources: Xataka.