SenseNova-Loop-9B plays its middle layers twice, and needs both passes

SenseNova-Loop-9B fits a real 20 GB card and answers right, at 2 to 3 tokens/s. Run its loop once and it doubles speed and breaks.

Published · By InferenceKey · 5 min read

Thirty-two layers stored, fifty-six layers run for every word. That is the trick inside SenseNova-Loop-9B-Preview, an open model from SenseTime that reads images, calls tools and ships under Apache 2.0. It reuses its middle layers instead of adding new ones, so it needs the memory of a 9B model. We ran it uncompressed on a real 20 GB RTX 4000 Ada: it read a chart, solved a word problem and called a tool, all correctly. The price is speed, 2 to 3 words a second on the authors' code. And the loop is not optional: when we made the model run its middle layers once instead of twice, it wrote twice as fast and stopped making sense. For anyone who deploys models, that means a multimodal open model on a workstation card, today, for short prompts and patient jobs.

Same SenseNova-Loop-9B weights, same word problem: three painters paint 2 fences in 5 hours, how many hours do 6 painters need for 8 fences? Left, the trained setting, 2 passes through the looped block, 56 layer runs per word: 277 tokens, 2.34 tokens per second, 118 seconds, a green box on the step that gives 10 hours and a big green Answer: 10. Right, the loop switched off, 1 pass, 32 layer runs per word: 1,024 tokens (the limit), 4.7 tokens per second, 218 seconds, red boxes where it drifts into Chinese and starts repeating one word, and a big red no answer

A repeat sign over the bridge

The model is a stack of 32 processing steps, called layers. Layers 4 to 27 run once, then again, with the very same learned numbers. Think of a band chart with a repeat sign over the bridge: the band plays the bridge twice, and the sheet music gains no pages. SenseTime calls it "~9B total parameters; ~14.6B inference parameters", their own count of the work per word, not one we checked.

It was trained on texts of up to 262,144 tokens, and the authors report 40.45 on Terminal-Bench 2.1, a test of agents working in a terminal, against 29.20 for Qwen3.5 9B. We did not reproduce those numbers.

The catch for anyone who wants to try it this week: the usual tools have no code for the repeat sign yet. The official llama.cpp cannot load it, and its model page says LM Studio and Ollama do not support it either. vLLM and SGLang support exists only as open pull requests, not releases. So the only way to run it on one card with both text and images is the authors' own Python code, shipped with the weights. That is what we deployed on InferenceKey.

What our test found: the second pass is what keeps it sane

We sent the same three requests to a 48 GB A40 on our cloud and to the real 20 GB RTX 4000 Ada. Both cards gave the same right answers. On a made-up bakery electricity bill, the model found August as the lowest month at 2,140 kWh, worked out 3,110 kWh saved and priced it at 653.10 euros. The tool call came back as a clean exchange-rate lookup.

Then we added a switch to our loading code that runs the middle layers once instead of twice. Same file, same question. Speed went from 2.34 to 4.7 tokens per second. On the fence problem the model went wrong in its first sentence, drifted into Chinese and repeated one word until it hit our 1,024-token limit, with no answer. On the chart it misread a number and answered 1,414 kWh and 293.34 euros, both wrong. Trimming the loop buys speed you cannot use: the model was trained to expect that second pass.

2 passes (as trained)1 pass (loop off)
Layer runs per word5632
Speed on the A402.34 tokens/s4.7 tokens/s
Fence problemAnswer: 10, right, in 118 sno answer after 1,024 tokens, 218 s
Chartright1,414 kWh and 293.34 EUR, wrong

Slow because of the software, more than the card

With the authors' code, a chart answer with thinking took about three minutes. What holds it back looks like software, not the GPU: one CPU core sat at 100 % handing the GPU small pieces of work, while the GPU was busy only 38 to 53 % of the time on the A40. Every word pays for 56 layers' worth of that handing out.

The unreleased SGLang pull request, which we built on the same A40, ran at 9.1 to 9.2 tokens per second. The word problem that took 101 seconds through the authors' code took 33.6 seconds there, with identical text. The built-in speed-up the model page lists as "Available", a small part that guesses the next few words, did not run on any engine we tried on one GPU.

Who should care

  • Teams that want an open model that reads images and calls tools on one small card. A real 20 GB card ran it uncompressed, with right answers, for prompts up to about 3,000 tokens. A prompt of about 6,000 tokens ran out of memory, and the server kept serving the next request.
  • Teams with long prompts. Plan on 24 GB or more: a 12,046-token prompt passed when we capped the A40 at a 24 GB budget.
  • Anyone who needs chat speed. Not yet. Watch the SGLang pull request.
  • Anyone tempted to cut the loop for speed. Don't, per the table above.

The honest limit: we ran each request once, one at a time, with prompts no longer than 12,046 tokens and no normal 9B on the same card to compare, so this shows the model works end to end, not how good it is or how much of the slowness is the loop itself.

The memory ladder, the SGLang build, the speed-up attempts and the example that runs it on your own worker are in the full SenseNova-Loop-9B test.