Groot-19B-A3B holds a 205K-token prompt on one RTX 5090, with under 600 MiB to spare
Groot-19B-A3B FP8 on one 32 GB RTX 5090: the full 262,144-token window, 262 to 304 tokens/s, and no start under a 24 GB cap.
Two hundred and five thousand tokens of prompt, one RTX 5090, and under 600 MiB of memory left over. That is Groot-19B-A3B, a new open agent model from GrootTeam, running its full 262,144-token window (the "256K") on a single 32 GB card on our cloud. We used the authors' 20.9 GB compressed build (FP8) on stock vLLM. It found a sentence we hid 60% of the way into that prompt. It wrote 262 to 304 tokens per second for one request, and its tool calls came back as clean JSON. For a team that deploys its own models, that is an agent with a quarter-million-token window on one gaming-class card you can own. The catch is the size of the card. When we capped the same GPU at 24 GB, vLLM refused to start, with room for about 9,600 tokens. Plan on 32 GB.
Same weights, same command, same card. With all 32 GB it answered a 205,525-token prompt right. Capped at 24 GB, it would not start.
A 19B agent model cut from a 35B parent
GrootTeam started from Qwen3.6-35B-A3B, the Qwen team's open 35B model, and cut it to 19B. The authors' page does not say how. They retrained it for agents on a small amount of open data, fewer than 0.1 billion tokens. They published four builds on 23 October, and we spotted it in AI Search's news video of 1 November.
It works like a large team where only a few specialists handle each word. About 3 billion of its 19 billion parameters work on each one. So it writes fast, but the GPU must hold all 19 billion.
The authors put it ahead of its own parent. On their table it beats Qwen3.6-35B-A3B and Gemma-4-26B-A4B on 12 of 16 benchmarks. On Terminal-Bench 2.1, where an agent works in a command line, they report 52.1 against 43.8 and 26.5. Those are their numbers, against open models of its size only; we did not reproduce them. The license tag says Apache-2.0, which allows commercial use, though the linked LICENSE file is missing.
The model is GrootTeam's work. Our part was the deployment, the tests and the measurements.
The full window fits in 32 GB, and 24 GB is not enough
The authors estimate, from memory sizes, that this build runs the full 256K context on one RTX 5090. So we went straight to that card.
Memory on a GPU pays for two things. One is the model's weights, a fixed cost. The other is a notepad that holds the conversation, and it grows with every token of context. Groot keeps Qwen3.6's layout, so its notepad stays as small as its parent's. What frees the room is the weights, 19B instead of 35B.
On the 32 GB card, vLLM had 6.8 GiB left for the notepad after loading the weights. That is enough for 307,024 tokens: the full window plus about 45,000 to spare. Then the hidden-sentence test. We buried the access code for a warehouse 60% of the way into a 205,525-token prompt. The model gave back 4471-HERON. The first word came after 34.6 seconds, the full answer after 40.3. At the peak the card held 32,016 of its 32,607 MiB. Picture a tour van with the amp wedged against the door: it rolls, but nothing else gets in.
| Memory budget on the same RTX 5090 | What happened |
|---|---|
| 32 GB, the real card, 262,144 tokens | started; the 205,525-token prompt answered right |
| 24 GB, emulated, 65,536 tokens asked | did not start; 9,648 tokens at most |
| 16 GB, emulated, the 13.5 GB build | did not start; a software fault at startup |
Under the 24 GB cap, the weights alone took 86% of the budget. That left 0.48 GiB for the notepad. Around 9,600 tokens is too short for an agent that reads files and tool results, and the authors themselves recommend keeping at least 128K. The 16 GB run crashed for a software reason, not for lack of memory, so it says nothing either way about the authors' claim for that build. Both caps were emulated on the same 5090, not real smaller cards.
Fast, clean tool calls, and a long think before every reply
One request through our API ran at 262 to 304 tokens per second. A 4,457-token answer took about 17 seconds, and the reply started in under a second. Part of that speed comes from a small extra layer the authors ship. With that layer off, vLLM wrote 170 to 211 tokens per second.
Tool calling worked the first time. Asked whether Lisbon or Oslo was warmer, it called our weather tool for both cities in one turn, then answered: "Lisbon is warmer right now with 19°C compared to Oslo's 4°C". The whole exchange took 1.81 seconds, in the shape an agent framework expects.
Budget for the thinking. Groot reasons before every answer, and that reasoning counts as output. "The capital of Australia, in one word" took 111 to 131 tokens before "Canberra". The authors turn thinking off with a per-request setting, and our API does not pass that setting through yet. Status: open on our side. The workaround is to start the server with thinking off by default, a one-line flag shown in our example's README.
On InferenceKey it needed no custom engine: the stock vLLM backend and the authors' settings. It was ready 611.5 seconds after the deploy call on a fresh machine.
Who should care
- Teams that want a long context on hardware they own. One 32 GB RTX 5090 ran the full window at this speed. One agent gets the full window; several agents at once must each take a shorter one.
- Teams on 24 GB cards. Skip the 20.9 GB build. The authors suggest their 11.7 GB one, which we did not test.
- Teams that need quick chat replies. Budget for the thinking that comes before every answer, and set a generous token limit.
The honest limit: one GPU, one request at a time, our own prompts and a few runs each, so this checks fit and speed, not its skill across a multi-step agent session.
Read the full test
The Research post has the memory numbers, the speed runs, the authors' benchmarks, every rough edge with its status and the SDK example for your own GPU. Read the full Groot-19B-A3B test in the Research post.