A 125B coding model on one RTX 5090, pushed to the limit

The eighth request went in, and every developer's answer slowed to 19 tokens per second. That was the point where one RTX 5090, a gaming card you can rent for 0.88 dollars an hour, stopped feeling like a private coding assistant and started feeling like a shared one.

We wanted to find that point. The card was running Strata, an open source engine that runs Qwen3.8-Flash-Next, a 125B parameter model, on consumer hardware. The question was simple: how many developers can one card serve before the wait becomes annoying, and how does that compare with paying for Claude Code?

Here is what we measured, on 17 October 2026:

The short version: one developer flies, two or three hold up well, eight is too many. The video shows the whole run, and the rest of this post explains why the numbers fall where they do and what they mean for your subscription bill.

How a 58 GB model runs on a 32 GB card

The model does not fit on the card. Strata gets around that with a property of the model itself.

Qwen3.8-Flash-Next is a mixture of experts. Think of it as a large team of specialists: for every word it writes, a router picks ten of the 512 specialists in each layer, plus one that always works. The rest of the team waits. Only about 6B of the 125B parameters do work for any given word.

The Coder version we used, published by the Deep Algorithms and Systems Lab at IST Austria, goes further. It keeps 256 of the 512 specialists per layer, chosen for code, and compresses them to about 1.9 bits each. That is how 125B parameters become a 58.4 GB file.

Strata then splits the work between the card and the computer. The system memory holds every specialist. The ones that get called most often move up to the card, which is much faster. When a word needs a specialist that is not on the card, the processor computes it at the same time, so nobody waits for a copy. On the 5090, 12,152 specialists sat on the card, about 30.7 of its 32 GB, and almost every call found what it needed there.

A second trick makes a single user fast. A small helper guesses the next four words, and the big model checks all four in one pass, keeping the right ones. In our run about three guesses out of four were right, which is where most of the 144 tokens per second comes from.

Adding developers until it bends

We sent real coding requests, each one with actual source code as context, first one at a time, then four at once, then eight.

One developer gets 144 tokens per second. That is several times faster than anyone reads, and the first word shows up in under two seconds.

Four developers get 38 each. The total, 125, is lower than with one. When Strata serves several people it gives each one a slot, and the slots do not use the guessing trick, so every word comes out one at a time. Each person still gets text faster than they can read it. For a small team that works.

Eight developers get 19 each. The card is at 98% and the total stays around 130. The first word now takes two to eight seconds, because the prompts are read one after another. It works, but everybody feels it, especially on long answers. A 4,000 token answer takes about half a minute for a developer alone on the card, and three and a half minutes when eight share it.

So the card does not run out of tokens. It runs out of seats.

A Claude Code subscription, or a card by the hour?

This is the comparison that made us run the test. A rented card is a strange kind of subscription: it has no token quota at all. It is like a rehearsal room rented by the hour. You can play as loud and as long as you want while you are in there, but only so many musicians fit.

Claude's own plans work the other way round. Pro costs 20 dollars a month, Max 100 or 200 dollars per developer, and a Team Premium seat 125 dollars a month. All of them include Claude Code, and all of them have usage limits that reset every five hours, plus weekly limits. Heavy agent users hit those limits, and past them they buy extra usage at API rates.

The card's math, at 0.88 dollars an hour:

For two or three developers who share the card during working hours, that is 50 to 80 dollars each, with no quota and no code leaving the machine. For one developer it is about the price of the top Max plan.

Now the honest part. It is not the same model. Claude is a stronger model and a whole product, with nothing to install or babysit. The Coder build we ran solves 75.6% of SWE-bench Verified according to its authors, against 82.8% for the full model it was cut from, which is 91% of the original. We did not measure that score ourselves. And a rented card is a rented card: the price and the quality of the host machine vary from one listing to the next.

Claude Code, pointed at the card

The part that surprised us most was how little changed for the agent. Claude Code talks to Strata through Strata's Anthropic-compatible endpoint. Two environment variables point it there:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
export ANTHROPIC_AUTH_TOKEN=local
claude --model qwen3.8-flash-next-coder

We asked for a complete Snake game in one file: neon look, saved high score, rising speed and a demo mode where the snake plays by itself. The model thought for about half a minute, wrote 408 lines in a single Write, then noticed a timing bug in its own code and fixed it with a one line edit that nobody asked for. The game worked, autopilot included.

One thing to know: with the tool list Claude Code sends, the model sometimes claims a task is done without calling the Write tool. Asking for the Write tool by name in the prompt fixed it every time in our sessions.

Who did the work

The model is from the Qwen team at Alibaba. The 1-bit Coder build comes from DASLab at IST Austria (Alireza Dadgarnia, Soroush Tabesh, Mahdi Nikdan, Michael Helcig, Eldar Kurtic, Maximilian Kleinegger and Dan Alistarh, with their GSQ and RCO methods). Strata, the engine that makes it run on a gaming card, is by Niko1221 and its community, under the MIT license. Our part was installing it from scratch on a rented card, loading it with developers until it bent, and recording what happened.

The question we are left with is not whether one card can serve a team. It can serve a small one. It is how many of your developers are actually generating at the same moment, because that number, and not the token count, decides the size of the card.

For engineers

• Hardware: one RTX 5090 (32 GB, PCIe Gen5 x16) on vast.ai at 0.88 dollars per hour, 16 core / 32 thread host CPU. One machine, one day of tests.
• Software: Strata 0.1.44, ./setup.sh --family coder --context 262144 --kv int8 --vision no. Model Qwen3.8-Flash-Next-GSQ-RCO-Coder IQ1_M (58.4 GB in two shards).
• Speeds: 1 client 144 tokens per second with speculative decoding (about 75% of draft tokens accepted). 4 parallel slots: 38 per client, 125 total. 8 slots: 19 per client, 130 total, first token 2 to 8 seconds. Parallel slots run without speculative decoding.
• Long context: prompt reading at about 4,500 tokens per second, so a 200,000 token prompt takes about 43 seconds, and single-user generation stays at about 165 tokens per second at that length (measured on an earlier host the day before).
• Cost per million generated tokens at full load: 1.70 dollars (1 client), 1.96 (4), 1.88 (8). Only valid while the card generates all the time.
• Two settings that mattered on rented containers. With 8 slots, keep VRAM headroom with --vram-reserve-mib 1536, or the driver starts paging GPU memory. On containers with a CPU quota (check /sys/fs/cgroup/cpu.max), set --pool-workers to a few less than the quota: by default Strata sizes its worker pool for every thread it can see, and the throttling cut generation from about 170 to 30 tokens per second on one host.
• Limits: one card model and one host for the published numbers; quality figures come from the model card, not from us; the Claude Code test is one task, a game in a single file, not a large repository.