Carbon-X reads DNA on a 24 GB card: labels 3 in 4 right, not the reasons

Carbon-X read 55 letters of a human gene carrying the one change behind sickle-cell disease, and flagged it as pathogenic. On one of our two cards, it then explained its answer by pointing at the wrong letter.

Carbon-X is Hugging Face's new open model that reads raw DNA inside an ordinary chat and answers in English. We served it with unmodified open software on an RTX 4090, a 24 GB gaming card, on our cloud. On that card it correctly called 75% of 40 known DNA changes from ClinVar, a public gene database, harmful or harmless (authors: 74.9%). It answered a pair of 4,096-letter sequences in about 2.5 seconds. If you run your own GPUs, this is a DNA specialist that keeps unpublished sequences on your machine and runs on standard open serving software with one command. Use its harmful or harmless labels as a first pass. Do not trust its explanations.

The image shows a single run on each card. Both got the label right, and neither named the gene or the protein change. Over 20 tries on each card, the answer was pathogenic 16 times and benign 4 times. The 48 GB card named the wrong base in 12 of 20 tries. The runs with shrunk weights got it right in 16 and 18 of 20. When we removed the model's built-in randomness, the 48 GB card said benign.

A chat model that reads DNA six letters at a time

Carbon-X is not a new kind of model, and that is why it was easy to serve. Under the hood it is Qwen3-14B, the open 14.8-billion-parameter chat model from the Qwen team, retrained on DNA and biology text. Hugging Face added one thing on the way in: a new way of cutting DNA into pieces before the model reads it.

Language models read text as tokens, chunks of a few letters each. Qwen's usual cutting step chops DNA into odd fragments, so long sequences cost many tokens. Carbon-X adds 5,515 DNA tokens to the ones Qwen already knows. Most of them (4,096) are six-letter blocks of A, C, G and T, so one token covers six bases. Think of a guitarist who reads a six-note run as one shape instead of six separate notes.

Six letters per token is why long DNA fits. In our test, two 4,096-letter sequences plus the question came to 1,445 tokens. The authors' own figure shows a short question with a 27-letter sequence taking 20 tokens with the Qwen3 tokenizer and 13 with Carbon-X's.

You do not have to mark the DNA yourself. Any run of six or more A, C, G and T letters in your message is marked as DNA automatically. Paste a sequence into a sentence and ask: is this variant harmful, what does this region do, which bases come next.

The catch for serving is that this cutting step is custom Python code that ships with the model. You have to let the serving software run Python from the model repository on your own machine. We read that file first: it uses only standard, widely used Python libraries. On first load it also downloads Qwen's base vocabulary from Hugging Face, so the machine needs internet access then. The rest is a stock 14B model.

Hugging Face says 66.7%, close to GPT-6.1 Sol at a sliver of the cost

The authors' headline, in their release blog and on the model card, is an average of 66.7% across five DNA benchmarks. GPT-6.1 Sol scores 67.5% on the same table and Opus 5.5 scores 60.9%. Hugging Face priced its own model at 5 dollars per GPU-hour, the others at retail API prices: for every 1,800 dollars spent on Sol, Carbon-X spends about one. The blog's subtitle says 1,000x instead.

The most concrete claim is about ClinVar, the public database where labs file genetic variants and what they think each one does. About 2.4 million of its variants are still of uncertain significance. The authors estimate that scoring all of them once with Carbon-X would cost about 300 dollars (their chart reads 271), against 32,000 dollars to 4 million with frontier models. We reproduced neither that estimate nor the 66.7%: their evaluation data is not public, so we built our own set from public sources.

We spotted Carbon-X in AI Search's AI NEWS roundup of 25 October. That video says Carbon-X outperforms GPT-6.1 on average. The authors' own table puts it 0.8 points behind Sol. The model card says llama.cpp and GGUF runtimes are not supported, yet the authors' own blog demo runs an 8-bit GGUF on a MacBook at 34 tokens a second. We did not test the laptop route.

Our test: two real cards, 120 public questions, the authors' own settings

We served Carbon-X on two cards on our cloud. A model is mostly its weights, the 14.8 billion numbers it learned. The main run used an RTX A6000 with 48 GB and kept each weight in 16 bits, the model's native format. The second run used an RTX 4090 with 24 GB and shrank each weight to 8 bits as it loads, nearly halving their memory. Both runs went through our platform, the way an application would call it.

The question set had 120 items, all from public data. Forty were real ClinVar variants in protein-coding genes, 20 harmless and 20 harmful, from the same test set the authors use. The model saw only the reference sequence, the changed one and our question, never the gene name. Sixty were GenomeQA multiple-choice questions on what a DNA region does, ten per task. Twenty asked for the next 24 bases of a human gene.

We used the authors' settings: no step-by-step reasoning before answering, and a little randomness in each reply. We also ran with randomness off.

One limit up front: 40 and 60 questions are small samples. On 40 variants, a score of 77.5% could honestly be anywhere from about 64% to 90%. Read our numbers as "matches the authors or not", not as a ranking.

Results: on real ClinVar variants, the labels match the authors

Carbon-X labelled 77.5% of our 40 ClinVar variants correctly on the 48 GB card with the authors' settings. The authors report 74.9% on all 1,018 variants of that test. On GenomeQA it scored 65.0%, against the authors' 63.6%.

Like for like, with randomness off, the 4090 scored 75.0% against 80.0%: two variants out of 40, inside the noise. Its answers matched the 48 GB card's on 36 of 40 variants and 57 of 60 questions. Turning randomness off changed one answer in 40. What limits these numbers is the small sample.

The errors lean one way. On the 48 GB card the model caught 19 of the 20 harmful variants but only 12 of the 20 harmless ones. So it flagged 8 of 20 harmless variants. On a long, real list, that is a lot of harmless cases for a human to review.

On the 4090, a full variant question on two 4,096-letter sequences comes back in 2.4 to 2.6 seconds, against 7.1 to 7.5 on the 48 GB card in 16-bit. Most of the gain comes from the 8-bit weights: with them, even the 48 GB card answered the same question in about 4 seconds.

Where it breaks: the label holds, the reasoning does not

The verdicts are usable, but the explanations around them are often wrong, even when the verdict is right. That matters because a chat answer in fluent English invites you to trust the reasoning.

On a real ClinVar variant where a T became a C, it gave the right label with the rationale "The variant changes C to C". Given 93 bases of the beta-globin gene, it translated them into protein correctly and then said the gene was PKM2, a different gene. The authors warn about exactly this: the model "can miscount bases, confuse coordinates" and may invent a sequence's identity.

Some tasks are weak too. On two of GenomeQA's six tasks it got 3 of 10 each, about as weak as the authors report (29.8% and 34.6%). Asked for the next 24 bases of a human gene, it got about 29% of bases right, where random guessing gets 25%. That set is ours, not the authors' NextBase benchmark (39.4%), so the two are not directly comparable.

It is not a clinical tool, and the authors say so.

Which GPU: a 24 GB RTX 4090 with 8-bit weights, not the 80 GB card on the model page

The smallest card that worked was a real 24 GB RTX 4090, with weights shrunk to 8 bits and room for 8,192 tokens per conversation. It peaked at 21.4 GB and answered faster than the 48 GB card (see Results).

In full 16-bit precision you need a 48 GB card. Our RTX A6000 showed 40.6 GB in use, and most of that beyond the weights is room reserved for the text of ongoing conversations. The 80 GB on the model card, for a 29.6 GB file, is generous.

Here is what ran on real hardware and what did not:

• Real cards: RTX A6000 48 GB in 16-bit (passed) and RTX 4090 24 GB in 8-bit (passed).
• Simulated by capping memory on the 48 GB card: a 24 GB cap in 8-bit passed (77.5% and 65.0%). Three 16 GB attempts failed: two would not start, and one managed roughly one token every five seconds.
• Not tried: 32 GB and 80 GB cards, and several other 24 and 48 GB models (list in the appendix).

The 8,192-token limit on the 4090 is half of what we used on the 48 GB card. We did not measure speed on prompts longer than our variant pair, or anywhere near the model's 40,960-token limit.

On InferenceKey: one command line, ready in six minutes on a cold card

From a cold 48 GB worker with no vLLM and no weights on it, Carbon-X was serving on stock vLLM, the open serving software, in 356 seconds, with one command line. A domain model should not need a custom engine, and this one did not. Most of the wait was installing vLLM (138 seconds) and the engine warming up (136 seconds). Downloading 30 GB of weights took 22 seconds, and loading them took 30. The 4090 was ready in 262.5 seconds. Our SDK example deploys the model on your GPU machine, waits for it, asks two questions from the model card and removes it.

Two settings did the work: a fixed vLLM version the tokenizer is known to work with, and step-by-step reasoning switched off on the server, as the authors trained it.

One problem shows up in the output. Answers that contain DNA come back wrapped in the literal markers <dna> and </dna>, and vLLM's cleanup option does not remove them. Status: open. The example's README tells you to strip the tags in your parser.

Should you use it? Yes for triage on your own card, no for its explanations

• If you need to sort many variants by likely harm, run it, but do not treat one answer as final. The sickle-cell verdict flipped to benign with randomness off. On the 4090, 120 questions sent 8 at a time took 45.9 seconds, a median of 3.3 seconds each. We did not measure tokens per second under load.
• If you need to know why, do not take its word. Positions, bases and gene names in its explanations were wrong too often. Compute positions in code.
• If your sequences cannot leave the building, it is Apache-2.0, not gated, and runs on one 24 GB card you control.
• If your card has 16 GB, skip it for now. No vLLM setup we tried worked.

To ask Carbon-X the authors' two DNA questions on your own NVIDIA card, run python main.py from the Carbon-X SDK example. It needs an InferenceKey project with an SDK token and an API key. It also needs an NVIDIA machine with a CUDA 13 driver (580 or newer) and 48 GB, or 24 GB with CARBON_FP8=1, plus access to Hugging Face on first load. It deletes the deployment when it finishes.

# 48 GB card, BF16 (main run)
vllm serve HuggingFaceBio/Carbon-X --trust-remote-code --dtype bfloat16 \
  --max-model-len 16384 \
  --default-chat-template-kwargs '{"enable_thinking": false}'

# 24 GB card, FP8 at load (what CARBON_FP8=1 sends)
vllm serve HuggingFaceBio/Carbon-X --trust-remote-code --dtype bfloat16 \
  --max-model-len 8192 --quantization fp8 \
  --default-chat-template-kwargs '{"enable_thinking": false}'

git clone https://github.com/inferencekey/inferencekey-sdk.git
cd inferencekey-sdk/examples/carbon-x-dna-chat-vllm-private-nvidia
cp .env.example .env        # INFERENCEKEY_SDK_TOKEN (ik_sdk_), INFERENCEKEY_API_KEY (ik_live_),
                            # INFERENCEKEY_PROJECT, IK_WORKER_ID, optional INFERENCEKEY_BASE_URL
python -m venv .venv && . .venv/bin/activate
pip install maturin
( cd ../../bindings/python && maturin develop --release )
set -a; . ./.env; set +a
python main.py                  # 48 GB card, BF16
CARBON_FP8=1 python main.py     # 24 GB card, FP8