Nanbeige4.3-33B on one card: 42 tokens/s on 24 GB, and on 16 GB only the 3-bit file works
A dense 33B agent model at 42 tokens/s on one RTX 4090. On 16 GB, the 3-bit file runs at 14 tokens/s; the 4-bit one crawls at 1.6.
With a dense model, the file that fits on the card beats the better file that does not. Nanbeige4.3-33B, a new Apache-2.0 agent model for coding, wrote about 42 tokens per second on one 24 GB RTX 4090 on our cloud, with valid tool calls every time. Under a 16 GB budget, its 4-bit file with a third of the model pushed off the card crawled at 1.6 tokens per second. The smaller 3-bit file, entirely on the card, passed every check, at about 14 tokens per second on a real 16 GB workstation card. For a team that deploys its own models: an open agent model on one card you own or rent, and a clear rule for picking the file. Try it on 24 GB, or on 16 GB with the 3-bit file. One catch first: with thinking on, the default, the agent's final answer came back empty in 2 of 3 runs.
A 33B where the whole band plays every bar
The agent models we tested before it, like Groot-19B and Clover-35B-A3B, use only a few of their parameters for each word: a big band where only a few soloists play each bar. Nanbeige4.3-33B is dense. All 33 billion parameters work on every token.
That has a price you can measure. To write each word, the card reads the whole model file from its own memory. With the 4-bit file, that is 20.8 GB per word. A smaller file means less to read and more words per second. Anything kept outside the card's memory is read far more slowly, and speed collapses.
The model went up on Hugging Face on 24 October, from the Nanbeige team at BOSS Zhipin. Its authors claim "state-of-the-art agentic performance among open models in the ~30B parameter class". Uncompressed, it is 66 GB, and their own serving recipes spread it over four GPUs. bartowski published compressed single-file versions for llama.cpp, the free engine we used.
The authors put it ahead of Qwen3.8-27B on two of three agent coding tests, and they print the loss too. These are their numbers, at maximum thinking. We did not reproduce them.
| Test (authors' numbers) | Nanbeige4.3-33B | Qwen3.8-27B |
|---|---|---|
| Terminal-Bench 2.1 | 80.9 | 73.0 |
| DeepSWE | 65.5 | 42.5 |
| SWE-Bench Pro | 60.3 | 61.7 |
What one 24 GB card gave us
With bartowski's recommended 4-bit file, the whole model sat on the RTX 4090. It wrote 42 to 44 tokens per second for one request, and the first token came back in a quarter to half a second on short prompts.
Our checks were small: a one-word answer, the 9.11 or 9.8 question from the model's page, a coding task with its own tests, and two tool calls. A tool call is the model asking your code to run a function, here a stock price and a weather lookup for two cities at once. The code passed its tests in 24.6 seconds. Every tool call came back valid, in the standard OpenAI format.
The same card held a 64K-token prompt, roughly 50,000 words, and found a secret word hidden at token 61,428. Writing slowed to 16 tokens per second as the prompt filled.
The 16 GB rule: smaller file, all on the card
The smallest card that passed is a real one: an RTX 2000 Ada, a 16 GB, 70-watt workstation card, with bartowski's 3-bit file. It passed the same checks at about a third of the 4090's speed, so a 400-token answer takes about 30 seconds instead of 10.
The two 16 GB rows in the picture ran on the full 4090 and peaked under the budget, so read them as memory figures, not speed promises. The lesson still holds: a third of a dense model off the card turned a fast model into one that took 815 seconds for a coding task. Going lower, the 2-bit file fit under 12 GB and got 9.8 right but explained its decimal place wrong, which is enough to keep it away from code.
The empty-answer problem
With thinking on, some answers arrive with an empty reply and the whole text sitting in the model's thinking field. It hit every 400-word essay we asked for, and the agent's final answer after the weather results in 2 of 3 runs, once on each card. The tool calls themselves were never affected.
With thinking switched off in a request sent straight to llama.cpp, the essay came back in the normal reply. Through InferenceKey, that per-request switch does not get through yet: our endpoint drops it. Status: open on our side. One server setting should turn thinking off for every request, but we have not run it, and the authors recommend keeping thinking on for agents.
On InferenceKey, the platform needed no changes: one version number and one launch command. The deployment was ready in 303.7 seconds on the 4090, almost all of it the worker (the small program on your GPU machine that takes orders from the platform) compiling llama.cpp for that GPU. On a throttled 16 GB host the same build took 19 minutes 50 seconds.
The model and benchmarks are the Nanbeige team's work, the compressed files bartowski's, the engine the llama.cpp maintainers'. Our part: the deployment, the tests and the SDK example.
Who should care
- Teams with a 24 GB card that want an open agent model for coding. Worth a try. Check the authors' scores against your own tasks.
- Teams with a 16 GB card. Use the 3-bit file entirely on the card, never the 4-bit one with a third off it.
- Teams running it as an agent today. Expect empty final answers with thinking on, until that switch gets through.
The honest limit: we ran one request at a time on our own small prompts, single tool calls and one agent turn, not an agent working through a real codebase.
Read the full test
The Research post has the memory ladder, every timing and the SDK example. Read the full Nanbeige4.3-33B test in the Research post.