Qwen3.5-0.8B: an AI edited 0.016% of its weights, and thinking loops fell from 93% to about 32%

Another AI changed 64 columns of numbers in each of two places inside Qwen3.5-0.8B, a small open reasoning model, and it did so with no training at all. We ran the edited model and the original side by side on one RTX 4090 on our cloud, on the same 50 math problems with the same settings. The edited one got 62% right. The original got 8%, because 96% of its answers never ended: it kept thinking in circles until it hit the limit of 32,768 tokens (pieces of words). If you serve small reasoning models on your own GPUs, that is a cost story as much as an accuracy story. An answer that never ends holds a slot on your card until the limit cuts it off, and it usually produces no answer. The edited file loads and serves with the same command as the base model. We checked only math with thinking on.

The question on the image is simple: how many divisors does 196 have? The answer is 9. The edited model found it and stopped. The base model started listing numbers past 3,300, one by one, and was still going when the limit cut it.

A model that cannot find the ending, and an edit that hands it one

The edit does one thing: it removes the habit that made the model loop. The math the model gets right was already in there.

Qwen3.5-0.8B is a member of the Qwen3.5 family with about 0.8 billion learned numbers (the weights). Before it answers, a reasoning model writes a scratchpad: it thinks out loud, then gives the result. Qwen's own model card warns that this size is "more prone to entering thinking loops" than its siblings. It is like a band that cannot find the ending of a song. Everyone is still in key, but the outro goes round and round until the sound engineer pulls the plug. On a GPU, the sound engineer is the token limit, and every token the model writes takes time on the card.

On November 2, 2026, a team at MIT CSAIL published a blog post on what they call weight-space programming. In one of their experiments, a large model they call GPT-6 Astra gets a tool to inspect and change the small model's numbers. It also gets practice problems, none from the test it is graded on. Its budget is a number of practice runs of the small model: 10, 25 or 50. Astra watches what the model does, reads its internals and edits the numbers directly. No training of any kind.

Of the base model's first 96 practice answers, 85 ran to the 32,768-token limit. With 50 runs, Astra found the pattern behind the loops in two places inside the model, and erased it from 64 columns of numbers in each. That is 131,072 weights out of the whole model: 0.016%. On MATH-500, a standard set of 500 competition math problems, the authors report a jump from 7.9% to 48.1% right. The share of answers cut at the limit fell from 93% to 33.1%. They report similar jumps on two other tests, which we did not rerun (see the appendix).

The authors published the exact change and a script that rebuilds it. The Hugging Face user akhaliq rebuilt the edited model file, reran it on all 500 problems and put up a public demo. His rerun got 51.0% right, with 31.6% of answers cut at the limit. We spotted the story in the AI Search news video of November 8.

62% right against 8% on one RTX 4090, and the loops fall from 96% to 20%

Under the authors' evaluation settings, the edited file stopped most of its loops on our hardware and got nearly eight times as many answers right.

Both models ran side by side on a single RTX 4090 with 24 GB on our cloud, each behind its own API endpoint. Both used vLLM, the open-source serving engine akhaliq's rerun used, with the thinking scratchpad on, and with the authors' randomness settings.

The questions were the first 50 MATH-500 problems in the order of akhaliq's rerun, one try each, up to 32,768 tokens per answer. All 50 ran at once for each model, both models at the same time. We graded them with the authors' own grading script.

Our 62% is above both published figures. That is the small sample talking: on these 50 problems, akhaliq's rerun averaged 54%, and our answers matched its usual result on 42 of 50. The base model's 8% and 96% sit right on the authors' figures.

The two times the base model finished an answer, it was right. Its other 2 right answers came from loops that had written the right number before the limit cut them.

The saving: 1.02 million tokens instead of 1.59 million for the same 50 problems

On the same 50 problems, the edited model averaged 20,444 tokens per answer and the base 31,832, and 20% of edited answers were cut at the limit against 96%. Per right answer: about 5 minutes of batch time against 32 seconds (398,000 tokens against 33,000). That gap rests on just 4 right base answers, so treat it as rough.

A GPU serves a fixed number of answers at once. With the base model, almost every one of those places was stuck in a loop, for up to 12.6 minutes on the divisors question. The edited model still writes long answers, but most of them end with a result.

Why not just lower the limit, or watch the stream and stop a loop, as Qwen's card suggests? That makes a loop fail sooner and saves card time. We have not tried it, or thinking off, on the base model, so test that before you swap files.

Where it stops working: it removes loops, it does not add skill

The edit fixed one bad habit. It did not make the model smarter, and the authors say so themselves.

They tried the same approach on Gemma 3 270M, a model that rarely loops: only 8.5% of its answers hit the limit. Astra could not improve it. It stayed between 6.8% and 7.7% on MATH-500 whatever the budget. In their words: "Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked." So do not expect this to work on a model whose problem is not loops.

Even edited, a fifth of our answers still looped, and 3 more ended without a final answer. The answers are long, so a single answer takes about a minute on its own. And the jump only came with the largest budget: with 10 or 25 runs, Astra's edits left the model at 12.3% and 14.1%.

The AI Search video also warns against the hype, and mentions an analysis that reportedly got the unedited model to about 44% on MATH-500 just by changing settings. The video names no source, and we have not checked it.

We tested math with thinking on only. We did not check general chat, following instructions, thinking off or images on the edited file. Qwen's card says the 0.8B model runs with thinking off by default.

Which GPU: the edited model finished under an emulated 6 GB memory cap

This check shows the model starts and finishes long answers under each cap. We used five problems it had already solved in the batch, and it gave the same five right answers under 6 GB as under 8 GB, in the same time. At 5 GB it passed only after we cut the room reserved for parallel conversations from 256 to 32. At 4 GB it would not start.

Every cap below 24 GB was a limit we set on the 4090. We did not run a real 6 or 8 GB card. A real small card also has slower memory, so expect it to run slower than this. We did not try the 4 GB cap with a shorter answer limit, or a compressed version of the model.

On InferenceKey: the base and its edited twin on one 24 GB card, ready in 7 minutes

One 24 GB card held the base and its edited twin at once, each given under half the card, with 16.9 GiB used at the peak. Both were ready 418 seconds after we asked for them, from a cold machine. About three minutes went to installing vLLM, and about three more to the engine's own warm-up. Once that install sat on the machine, a new engine was ready in about one minute.

Our API cannot yet change some settings per request, such as the thinking switch, so we fixed them on the server for every caller.

One problem you will hit: a base-model answer streamed through the endpoint stopped after 152 seconds, with no error our client could see and no end marker. To a client, a cut answer looks like a finished one. We suspect the platform's two-minute silence timeout, which would send an error that our example script ignores, but the cause is not confirmed. Until then, check that each streamed answer says it finished properly before you trust it.

Should you use it? Try it if you serve Qwen3.5-0.8B with thinking on

• If you already serve Qwen3.5-0.8B with thinking on and send it math: try the swap. It is the same size and license, and runs on the same engine.
• If you want the best math from a tiny model: treat it as a fix, not an upgrade. It brings out what the model already knew, and it still writes long answers.
• If you are sizing a GPU: the 6 GB result was emulated on a 24 GB card, not a real 6 GB card; see the memory section.
• If you run another small model that loops: this is a published method with logs, not a product. The Gemma result says it only helps when loops are the problem.

To run the same comparison on your own GPU machine, use the Qwen3.5-0.8B weight edit SDK example with IK_WITH_BASELINE=1. It needs a registered NVIDIA machine (a worker) with a 24 GB card and about 15 GB of free disk. The access keys and the local SDK build are in the step by step below. It deletes both workloads when it finishes.

vllm serve <model> --dtype bfloat16 --max-model-len 34816 \
  --gpu-memory-utilization 0.45 \
  --default-chat-template-kwargs '{"enable_thinking": true}' \
  --override-generation-config '{"temperature": 0.6, "top_p": 0.95, "top_k": 20, "presence_penalty": 1.5}'