Clover-35B-A3B writes 208 tokens/s on one 24 GB card, as an ordinary model for now
DiffuSpace's diffusion LLM in stock llama.cpp on one RTX 4090: 208 tokens/s and 128K context after a 17 s patch, without the parallel mode.
Two hundred and eight tokens per second, on one RTX 4090 on our cloud. That is what Clover-35B-A3B-Preview, DiffuSpace's open diffusion language model, gave us in stock llama.cpp v0.9.0, the free engine release that added it this week. It also read prompts up to its full 128K window (a short novel) in 21.4 of the card's 24 GB. For a team that deploys its own models, that is a new open agent model on one card you own or rent, today. The catch sits in the release notes: support is "AR mode" only, so Clover writes one token at a time, like any other model. The parallel trick behind DiffuSpace's 515 tokens per second is not in that code. And the official file needs a 17-second patch before it answers anything sensible. Try it on a 24 GB card for agent work. Do not buy hardware for the diffusion speed yet.
Built to fill in blocks, run as a typist
An ordinary language model works like a typist: one token, then the next, each waiting for the one before. Clover was trained to work more like someone filling in a crossword. It lays out a block of 32 empty slots, fills every slot it is at least 95% sure of in one pass, and comes back for the rest. Fewer passes mean more text per second, if the software does the crossword.
Clover is DiffuSpace's first release, derived from Qwen3.6-35B-A3B, an open model from Alibaba's Qwen team. Of its 35 billion parameters, about 3 billion work on each token: 256 specialists, and each word consults 8 plus one generalist. The license is Apache-2.0, so commercial use is fine.
DiffuSpace's blog reports 515 tokens per second on one H200 data-center card, against 365 for a sped-up Qwen model of similar size, and 680 on a GB300. Both used the full uncompressed model, with software and mode unnamed.
Here is the gap. The llama.cpp support, pull request #31306 by Maphist0, was merged on 2 November and shipped in v0.9.0 on 4 November, in AR mode: the typist. The faster modes live elsewhere. DiffuSpace's own Python code needs about 69 GB for the weights alone, so an 80 GB card. An SGLang pull request by Zirui Wu (#46927) adds a guess-then-check mode and reports 453.59 tokens per second on one H200, but it is still open, waiting for review. We measured what one card gives you today.
The official file needs a 17-second patch first
As published, llama.cpp v0.9.0 refuses to load the file: key not found in model: clover_moe.expert_shared_count. We added just that label. The model loaded, and asked for the capital of Australia, it answered with 600 copies of the letter G, one note held like guitar feedback.
DiffuSpace wrote the file with its own llama.cpp build, which differs from upstream in three small ways. Our script, patch_gguf.py, fixes all three and copies every other byte as is, so nothing is re-compressed. It took 17 seconds on the 19.6 GB file. After it, the answer was Canberra. Status: fixed in our SDK example, still open in DiffuSpace's files and in llama.cpp.
About 208 tokens per second, and the whole 128K window fits
With the patched file, Clover wrote 202 to 211 tokens per second for one request, and the first token came back in 0.13 to 0.31 seconds.
| What we asked | What came back | Time |
|---|---|---|
| A compound drone dispatch message, five orders | 5 of 5 right, as clean JSON, every run | 2.1 to 2.3 s |
| "Is it warmer in Lisbon or in Oslo?" with a weather tool | Two tool calls at once, both valid | 1.29 s |
| A whole desktop in one HTML file | A working page with a calculator that says 7 x 6 = 42 | 20.3 s |
| A code hidden in an 89,367-token prompt | Found | 13.45 s |
DiffuSpace's own drone demo took 0.97 seconds, on bigger hardware in an unnamed mode: not a like-for-like race.
Smaller cards worked too, slower. llama.cpp can keep some of the specialists in the computer's main memory instead of on the card. With 10 layers of them moved out, the model fit in 15,456 MiB, a 16 GB budget, and still wrote about 102 tokens per second. We emulated that budget on the 4090: a memory figure, not a speed promise.
One rough edge: Clover always thinks before it answers, and our endpoint does not yet pass through the switch to turn that off (open on our side).
On InferenceKey, no platform code changed. We pinned the worker (the small program on your GPU machine that takes orders from the platform) to llama.cpp v0.9.0. It built that release in 4 min 50 s the first time, and after that a ready endpoint took 11.7 seconds. From there it is the same OpenAI-style API as every other model.
The model and benchmarks are DiffuSpace's work, the llama.cpp support is Maphist0's, the SGLang branch is Zirui Wu's. Our part: the patch, the deployment and the measurements.
Who should care
- Teams that want an open agent model on one 24 GB card. Worth a try. Patch the file first, and compare it with its Qwen base on your own tasks.
- Teams with a 16 GB card. Probably, with specialists in RAM, at about half speed.
- Teams that want the diffusion speed from the blog. Not on llama.cpp today. Watch the SGLang pull request, or use DiffuSpace's Python code on an 80 GB card.
- Teams that pick models by benchmark. Those scores are the vendor's, and may not hold in one-token mode.
The honest limit: we did not run the Qwen3.6-35B-A3B base on this card or test a real 16 GB card, so we cannot say whether Clover in this mode beats its base.
Read the full test
The Research post has the memory ladder, every timing and the SDK example. Read the full Clover-35B-A3B test in the Research post.