Nemotron 3.5 Transcribe on one 32 GB card: a prompt picks the transcript, 3 minutes in 4.4 seconds

Ask Nemotron 3.5 Transcribe for speaker tags. A three-minute doctor's visit comes back in 4.4 seconds of GPU time, split into doctor and patient, with a start and end time on each of its 561 words. Ask for a clean transcript and the same model drops the "um"s instead.

We ran NVIDIA's new open speech model on one 32 GB Blackwell graphics card, with the open-source software NVIDIA ships. For anyone who runs their own GPUs, it means one deployment hands back the transcript each team needs, chosen per request. On 40 standard test clips it got 6.82% of words wrong, in the same range as the models we compared. A 24 GB share of a bigger Blackwell card also ran it, four to eight times slower. Our SDK example deploys it on your own Blackwell GPU.

Same recording, same card, two prompts. The clean version on the right adds speaker tags nobody asked for.

One model, and the instruction decides what kind of transcript you get

Nemotron 3.5 Transcribe works more like a chat model that listens. You send the audio plus a sentence of instruction, and the instruction picks the transcript.

NVIDIA's model page lists what that sentence can ask for. Word for word, with every filler. Cleaned up for reading. Tagged by speaker, with a time for every word. Translated. It can also name the language or answer a question about the audio. It covers 40 languages and recordings up to one hour.

For a team, that is the difference between running three systems and running one. A medical scribe wants the clean version, a compliance team wants every "um", and a meeting tool wants who said what. Without it, speaker labels and cleanup are separate steps you add yourself. Here it is closer to a studio that records one take and mixes it differently for each listener: one loaded copy, a different sentence in each request.

The model is big for speech. It has 34 billion parameters in total, but only about 4.3 billion work on each piece of audio. Picture a large team of specialists where each word consults a handful of them while the rest wait. That is why it runs fast for its size.

NVIDIA released it on 28 October in three file sizes: 67.5 GB, 34.7 GB and 21.6 GB. The smallest file only runs on NVIDIA's newest generation of cards, Blackwell. NVIDIA says it is almost as accurate: 3.6% more errors than the largest version.

The license is OpenMDW-1.1. It lets you use, change and share the model without restriction, keeping the notice. It puts no conditions on what you do with the transcripts. NVIDIA's page states plainly that commercial use is allowed.

NVIDIA built the model and its open serving recipe, and open-source projects built the engine it runs on. We spotted it in the AI Search news video of 1 November. We packaged it, ran it on cards a team can rent and counted the errors. The appendix credits everyone by name.

NVIDIA's numbers are strong, and its fastest figure is not open yet

On NVIDIA's own tests the model leads the open speech models it compares against. On nine standard English test sets it averages 4.71% wrong words, against 4.94% for Qwen3-ASR-1.7B. In seven other languages it scores 4.62% against 6.49% for the same Qwen model. When people talk over each other, it gets about 16 words in 100 wrong or given to the wrong speaker (15.80), against 16.83 for the strongest comparison NVIDIA lists. Those are NVIDIA's runs of the largest version, captured on 12 October. We did not reproduce them.

The speed claim needs a closer read. NVIDIA's page says one 96 GB RTX PRO 6000 Blackwell transcribes up to 2,238 seconds (about 37 minutes) of audio every second. The page adds that this figure comes from NVIDIA's own packaged engine. Those optimizations "are not yet part of the open-source release". The open version shipped on release day "is expected to be roughly two times slower". We used the open version, so our numbers come from that slower path.

One more limit, also from NVIDIA's page: this is an offline model. You hand it a finished recording and get a transcript back. It does not caption a live call as people speak.

The test: one 32 GB RTX PRO 4500 Blackwell and the open software

What does one 32 GB Blackwell card give you with the software anyone can download? NVIDIA's page lists the RTX 5090, a 32 GB gaming card, as compatible with the small file. No RTX 5090 was available on our cloud that day. We used an RTX PRO 4500 Blackwell, a workstation card with the same 32 GB and chip generation. Its memory is about half as fast as a 5090's. We did not measure a 5090.

We fed it three kinds of audio:

• the first three minutes of a PriMock57 consultation, a staged doctor-patient visit with a human transcript
• five minutes of earnings-call audio, stitched from 33 segments of a public test set
• the same 40 short English clips from a public speech-recognition benchmark that we used to compare Underdog Bark and Qwen3-ASR last week.

That gives a direct comparison with two models we already know. One of them, Qwen3-ASR, is the speech model we already run for our own transcription. The sample is small: 924 words of clips, one three-minute and one five-minute file, one request at a time.

Three minutes of a doctor's visit in 4.4 seconds, with who said what and when

Each prompt in the table took under five seconds of GPU time for three minutes of audio, about forty times faster than the recording plays.

Verbatim kept the hesitations a compliance team needs, and speakers split the turns. The turns line up with the human reference when you read them side by side, but we checked that by eye: we scored the words, not the speaker labels. A fourth prompt, "translate to Spanish", came back in Spanish after 4.9 seconds on the GPU. We did not score it.

The five-minute earnings-call file took 4.8 seconds on the GPU, in one request. It got 16.11% of the words wrong, but the stitched reference has gaps where the segments meet, so only the comparison carries weight: Bark got 18.69% on the same file in last week's test, and Qwen3-ASR, in last week's setup, stopped partway through it.

On the 40 short clips, the accuracy number that compares across models, it got 6.82% wrong. Bark 3.0 Pro got 6.28% and Qwen3-ASR-1.7B 7.03% on the same clips, scored the same way. So Nemotron sits between them, closer to Qwen. Those two ran on a different card in last week's test, so the error rates compare and the speeds do not.

Where it slips: the clean prompt over-tags, and meetings are its weakest set

The clean prompt is the one to test before you trust it. Our clean instruction says "omit fillers and accidental repetitions; keep the remaining spoken wording unchanged" and ends "Output: transcript only". It asks for no tags. The model removed every filler and also added 50 speaker tags in three minutes. It left a false start, "Hello, how.", that a reader would expect gone. If your pipeline expects plain prose, strip the tags or use the speaker prompt on purpose.

On the 40 clips it was best of the three on audiobook speech: 0.94% wrong, against 1.88% for Qwen3-ASR and 2.35% for Bark. AMI, recorded office meetings with people talking over each other, was its weakest set at 14.62%, behind Bark (11.54%) and ahead of Qwen3-ASR (16.92%). A meeting tool should check its own recordings before switching.

Which GPU: a real 24 GB slice worked, four to eight times slower

The smallest hardware that worked was a real 24 GB slice of a Blackwell card. The 32 GB card is the fast option, not the minimum.

That slice comes from an RTX PRO 6000 Blackwell, a 96 GB card that can be cut into isolated pieces. Each piece gets 24 GB of memory and a quarter of the card's compute. On it, the three-minute speaker run took 35.3 seconds instead of 4.4. That is eight times slower, and still about five times faster than the recording plays. Short clips were about four times slower, the five-minute file about seven.

The two setups that worked:

In its fast mode the model did not fit in the 24 GB slice. We did not try a full 24 GB Blackwell card, 16 GB, the mid-size file on a 48 GB card, or many requests at once.

On InferenceKey: one loaded copy answers every style

One running copy on one rented card answered all four styles at the same address, so a new transcript style costs a sentence, not another deployment. Through the platform, a restart was ready in 88 seconds. A fresh machine was ready in 217 seconds. Our very first start took about six minutes while one-time GPU code was built.

It took a custom backend, a short Python file that tells our platform how to install and run a model on each GPU machine. Our built-in option installs the serving engine alone and cannot add NVIDIA's speech add-on, so our file builds both from NVIDIA's recipe. Our backend exposes the standard OpenAI speech route, /v1/audio/transcriptions. Our Qwen3-ASR deployment answers on the same route, so the request a team already sends keeps its shape. The new part is the prompt field.

One limit you will hit is still open. Our transcription address takes files up to 2 MB, about eight minutes of compressed speech. The model takes an hour of audio, so cut longer recordings for now, or compress them hard. Each request is diarized on its own, so speaker labels may not carry over between pieces; we did not test split recordings.

Should you use it? Yes for several transcript styles from one Blackwell card

• You need different transcripts for different teams. One deployment gives verbatim, clean, speaker-tagged and translated text. Check the clean output for stray tags.
• You need the lowest error rate on English clips. It is in the same range as Bark 3.0 Pro and Qwen3-ASR-1.7B on our sample, not ahead. Its gain is the choice of output, not raw accuracy.
• You have a 32 GB Blackwell card. It runs fast. A 24 GB Blackwell slice also runs, slower.
• You run RTX 4090s or older cards. The small file will not run, it needs Blackwell. NVIDIA lists the mid-size file on a 48 GB L40S.
• You need live captions. Not this model, it is offline only.

Follow the Nemotron 3.5 Transcribe SDK example to get the three transcripts of your own recording on your own Blackwell GPU. Fill in .env, run python publish.py once and then python main.py. It needs a 32 GB card for the defaults, about 50 GB of disk and an audio file under 2 MB.

cp .env.example .env                # INFERENCEKEY_SDK_TOKEN, INFERENCEKEY_API_KEY,
                                    # INFERENCEKEY_PROJECT, IK_WORKER_ID, IK_AUDIO (< 2 MiB),
                                    # INFERENCEKEY_TENANT_ID, INFERENCEKEY_USER_TOKEN (for publish.py)
pip install inferencekey requests openai  # client side; requirements.txt is the backend's
python publish.py                   # once: packages dist/*.tar.gz and uploads it to your tenant;
                                    # without the two publish variables, upload that file from
                                    # Dashboard -> Custom backends
python main.py                      # ensure, wait, verbatim/clean/speakers on IK_AUDIO, delete

NEMOTRON_GPU_MEMORY_UTILIZATION=0.93
NEMOTRON_MAX_MODEL_LEN=16384
NEMOTRON_MAX_NUM_SEQS=2
NEMOTRON_MTP=0
NEMOTRON_ENFORCE_EAGER=1

from openai import OpenAI
client = OpenAI(base_url="https://cloud.inferencekey.com/endpoint/<project>/nemotron-transcribe/v1",
                api_key="ik_live_...")
out = client.audio.transcriptions.create(model="default", file=open("meeting.ogg", "rb"),
                                         prompt="speakers", timestamp_granularities=["word"])