Ming-3.0-tts clones a voice in about 3 GB of GPU memory, but speaks slower than real time

inclusionAI's MIT speech model on a 16 GB card: 2.9 GB peak, 17 of 19 lines word for word, clones that held, about 0.65x real time.

Published · By InferenceKey · 10 min read + technical appendix

We wanted to know whether a 3 GB speech model really runs on a small graphics card. Ming-3.0-tts-0.5B is an MIT-licensed model from inclusionAI, Ant Group's AI lab, and its authors only list tests on data-center cards with 80 GB and 96 GB of memory. On a 16 GB RTX 2000 Ada it needed 2.9 GB at its peak, and made the exact same audio when allowed only 3 GB, as on a small card. We typed a laugh and a sigh into a line of text: every word came out right, and the waveform shows bursts where the laugh tag sat. If you run your own GPUs, that means one small card can clone a voice, invent a new one from a sentence and speak Spanish as well as English. The catch is speed: about 1.5 seconds of work for every second of speech. So it suits recorded narration, not a live voice agent. Our SDK example deploys it on your own GPU in about a minute.

Two waveforms of the same cloned voice saying the same line on one RTX 2000 Ada 16 GB. Top, no tags: 5.3 s of audio in 8.1 s of GPU time, 0.66x real time. Bottom, with [Laughter], [Sigh] and [Uhm] in the text: 7.0 s of audio in 11.0 s, 0.63x. Amber boxes mark each tag: laughter bursts before 'Okay', only a faint breath at the sigh, and the 'Um' that Whisper heard

Same voice, same line, same card. The three tags in brackets added 1.7 seconds of sound. The words came out right both times.

Three numbers from our test of Ming-3.0-tts-0.5B: 2.9 GB peak GPU memory, and it still ran with PyTorch capped at 3 GiB. 0.65x real time on an RTX 2000 Ada and 0.76x on an RTX 4090: the decode loop sets the pace. 60 seconds from ensure() to serving with empty caches: venv, 3 GB of weights, load

What the model page promises: five ways to shape a voice in a 3 GB download

The model page lists five ways to shape a voice. It clones a voice from a short clip. It builds a new voice from a written description ("voice design"). It follows free-form directions on how to say a line. It ships built-in English and Chinese voices. And it takes 14 inline tags, words in square brackets such as [Laughter], [Sigh], [Cough] or [Uhm], that turn into sounds in the middle of a sentence.

The page lists Chinese, English, German, Spanish, French, Korean and Japanese, plus two Chinese dialects.

The credit for the model belongs to the inclusionAI team at Ant Group. They released the weights and the code under MIT, which allows commercial use, with their own benchmarks and an honest limitations section. On their tests, Ming gets about 1.6 English words in 100 wrong, between two rival open models (1.45 and 1.83). In Spanish it is about 1.1 in 100. At matching a written voice description in Chinese, it scores 72.3% against Qwen3-TTS's 77.1%. We did not rerun any of these.

We spotted it in the AI Search news video of 25 October. The presenter said the small model is "only around three gigabytes, so you should be able to run it on many consumer GPUs."

The model page says something narrower: "Tested on NVIDIA H800-80GB and H20-96G with CUDA 12.4." Those are data-center cards, and the authors give no memory figure and no smaller GPU. Our part was the rest: packaging their code, running it on a smaller card, and checking the output.

The test: one 16 GB card, 19 lines in three languages, checked by machine

We ran the authors' own code, unchanged, on one RTX 2000 Ada with 16 GB. It is a small workstation card with a 70-watt power limit. We sent 19 test lines in English, Spanish and Chinese, plus three controls: two lines without their tags and one with the numbers written as words. They covered cloning, voice design, free-form direction, built-in voices and tags, plus numbers and paragraphs of up to 21 seconds. Then we repeated five of them on an RTX 4090 to see whether a faster card speeds it up.

For cloning, we needed a voice nobody owns. So the reference was a synthetic 5.2-second support-desk line from a stock voice of Pocket TTS, a small speech model by Kyutai that we also run. That follows the authors' own rule: do not impersonate anyone without consent, and label synthetic speech as synthetic.

The checks were done by machine. Whisper, OpenAI's speech-to-text model, transcribed every clip. A speaker-recognition model scored how close each clone sounds to the reference, from 0 (unrelated) to 1 (identical). Nobody on our side listened to the clips, so we can tell you the words were right and the voice matched, not how natural it sounds. The reference was also the easy case: a clean synthetic clip, not a real voice from a laptop mic or a phone line. Each line ran only once.

Results: 17 of 19 lines word for word, and clones that keep the voice in Spanish

The text was almost always exact. Of 19 lines, 17 came back word for word in English, Spanish and Chinese, including paragraphs of 18 to 21 seconds. Both misses were numbers, covered in the next section.

The clones held, even across languages: from an English reference, the Spanish clone kept the voice as well as the English one.

ClipSame voice as the reference? (0 to 1)
English clone0.87
English clone with tags0.85
Spanish clone, from the English reference0.84
Built-in voice US_Emma (not a clone)0.54
Designed narrator voice (not a clone)0.32

One caveat: our scoring model is the one Ming itself uses to learn a voice, so it leans in Ming's favour. A second, independent checker agreed, but it is lenient: it also rated a different female voice close to our female reference.

The tags made real sounds, but only some can be checked. Tagged clips were 1.3 to 1.7 seconds longer than the same lines without tags. Whisper wrote "Um" where [Uhm] sat and "Huh?" for [Question-huh]. Laughs have no words, but the waveform shows them. The sigh barely shows. We tested tags in English only. The authors document them for English and Chinese text, so tags inside Spanish are untested.

Voice design got every word right. We asked for "a gravelly old sea captain in his seventies, amused and a little hoarse", among others, and every designed line came back word for word. The designed voices are clearly different from our reference (0.25 to 0.32). Whether each voice fits its description, we did not measure.

Where it breaks: numbers, and speech that comes out slower than it plays

Numbers are unreliable, and the model is slower than real time on every card we tried.

Numbers. The two misses came from the step that turns written text into words to say. In English, 4521 was read as "four thousand five hundred twenty, one", with a pause. A phone number, 555-0142, was read as "five hundred fifty-five minus one hundred forty-two". Spanish skips that step entirely. So "1.249,99 euros, un 12,5 %" came out garbled. Writing the numbers as words did not fix the English case. If a price or an order number matters, check the audio.

Speed. "Real time" means one second of audio made in one second. On the RTX 2000 Ada, a 20.7-second English paragraph took 31 to 32 seconds to make, and every line but one Chinese outlier ran at that pace. The RTX 4090, a far faster card, made a 20.9-second paragraph in 27 seconds. That is about 17% faster, and part of it may come from the 4090 machine's different processor.

The card was not the bottleneck. The small card was busy less than a third of the time, the 4090 a fifth at most. Ming writes audio in slices of 80 milliseconds, one after another. Between slices, a loop written in Python does its bookkeeping and hands the next slice to the card. In the time 12.5 slices play, it makes about 8.5: two thirds of the pace. Think of a session guitarist who waits for the bandleader to call every chord: hire a faster guitarist and the song still runs at the bandleader's pace.

The authors' roadmap marks support for a faster serving engine, vLLM-Omni, as done. We did not try that route, so whether it fixes the speed is open.

One request was much slower: the authors' own Chinese voice-design example took 50 seconds for 5 seconds of audio, because the model never decided one piece of text was finished. We did not see it in English or Spanish.

Which GPU: capped at 3 GB, the audio was identical

The smallest setup that worked: our 16 GB card with the model allowed only 3 GB of its memory, with default settings. The audio was byte-for-byte identical to the uncapped run, at the same speed.

The driver needs about 0.4 GB on top, so we estimate a 4 GB card is enough, untested.

No real card under 16 GB ran. We tried a 12 GB RTX 3080 Ti twice, and both machines came up without the remote access our setup needs, so neither ever ran the model. 8 GB cards were out of stock. A smaller chip could also be slower than ours, and we have no number for it.

On InferenceKey: ready in 60 seconds, with the authors' own code behind an API

On your own GPU you can run the authors' code as it is, without waiting for someone to package it, and ours was serving in 60.3 seconds. InferenceKey has no ready-made support for Ming, and the authors ship it as Python code, so we put that code behind an API. Our custom backend is a short Python file that InferenceKey installs and runs on your GPU machine. It fetches the authors' code at a fixed version, downloads the weights and loads the model once. Your app then calls it the way it calls OpenAI's chat API, and gets back the WAV inside the reply.

From a clean machine, about half of that time was installing the Python libraries. The 3 GB of weights took 7 seconds to download and under 3 seconds to load. With everything already on disk, a restart took about 13 seconds. The backend answers one request at a time, and we did not test several at once.

One rough edge you will hit too:

  • Apps built for OpenAI ask for speech at a separate address, and that address cannot return a custom backend's audio yet. So the example uses the chat address and reads the audio out of the reply's text. Status: open, tracked as IK-531.

Should you use it? Yes for recorded narration on a small card, no for live voice

  • If you make voice-overs, audiobook samples or narrated demos, it is worth running. One small card gives you cloning, designed voices and tags, and the license allows commercial use. The words and the voice match held up on machine checks. Listen to the samples above before you commit.
  • If you need a voice agent that answers in real time, look elsewhere for now. It makes speech slower than it plays, and a bigger card barely helps.
  • If your text has prices, phone numbers or dates, check every one in the audio.
  • If you need Spanish, it is usable for cloned narration, with numbers checked by hand. Our earlier Spanish Pocket TTS voice ran faster than real time on a desktop CPU (1.05x), Ming at 0.61 to 0.77x on a GPU. We have not compared their quality.
  • If your card is small, we estimate about 4 GB is enough for memory, untested (see the GPU section).

To hear four lines on your own NVIDIA GPU, publish the backend with python publish.py and run python main.py from the SDK example. You get a built-in voice, a designed voice, a clone with a laugh and a sigh, and the same clone in Spanish. You need an InferenceKey project with an SDK token and an API key. Your GPU machine needs an up-to-date NVIDIA driver (CUDA 12.4 or newer) and about 3 GB of free GPU memory.

For engineers

Model and files

Model card, GitHub (inclusionAI/Ming-3.0-tts), authors' benchmarks. Repo inclusionAI/Ming-3.0-tts-0.5B, created 2026-09-22, lastModified 2026-10-27T12:07:15Z, sha 7247f4c94a5b2e987404ed28a5cfec41b1a4ca61, license MIT, not gated, pipeline text-to-speech, 5,546 downloads and 156 likes on 2026-10-28.

  • model.safetensors: 2,959,055,504 B. Safetensors metadata: 1,479,454,868 parameters (1,479,454,804 BF16 plus 64 F32). campplus.onnx (speaker encoder): 28,303,423 B. Full repo about 3.0 GB, including a 390,569 B PDF (Ming-3.0-tts-0.5B-TDS2026.pdf) and figures.
  • Card table: dense, Qwen2.5-0.5B backbone, 0.5B parameters. The weights file stores about 1.48 billion in all; the authors do not break down where the rest sits. Also on the card: languages zh, en, de, es, fr, ko, ja, dialects Sichuanese and Cantonese, Python API ming_tts.MingTTS, CLI infer.py, 44.1 kHz.
  • The 14 tags: [Breathing], [Cough], [Sigh], [Laughter], [Surprise-ah], [Surprise-oh], [Dissatisfaction-hnn], [Uhm], [Confirmation-en], [Question-ei], [Question-ah], [Question-en], [Question-oh], [Question-huh]. Documented for Chinese or English text, case-sensitive, routed around text normalization (cookbooks/instruction_en.md, frontend/toolkit.py).
  • Tasks: TTS, FREEFORM, VOICE_DESIGN, MULTILINGUAL, BASIC, EMOTION, DIALECT, IP.
  • Hardware statement: "Tested on NVIDIA H800-80GB and H20-96G with CUDA 12.4. flash-attn is optional; inference falls back to PyTorch SDPA." No smaller GPU and no VRAM figure.
  • Code at commit 9332fb9443325d4726ce02cefd4ec1c211df4079 (2026-10-27, "docs: align recommended voices, catalogs and examples"), the commit our backend pins. Earlier: 028c1eac (2026-10-21, long-text splitting fix), e999c39f (2026-10-19, "mark vLLM-Omni support"). requirements.txt pins torch 2.6.0, torchaudio 2.6.0, transformers 4.46.3, accelerate 1.3.0, x_transformers 2.9.2, diffusers 0.33.0, soundfile 0.12.1, plus onnxruntime, inflect, einops. pynini 2.1.6 (text normalization) and flash_attn 2.7.4.post1 are optional. A Docker image is offered (torch 2.6, CUDA 12.4, transformers 4.46.3).
  • MingTTS loads in torch.bfloat16 (hardcoded) on cuda:0. synthesize() defaults: cfg=2.0, sigma=0.25, temperature=0.0, max_length=24 (fragment split), max_decode_steps=1000. Non-zh/en language_id text bypasses the FST normalizer ("Non-zh/en text is passed through unchanged").
  • Decode loop (modeling_bailingmm.py): one LLM forward plus a flow-matching head (FlowLoss.sample, 10 steps by default, with CFG) per step, until a stop head fires. The 512-position static KV cache caps the budget at 512 minus the prompt length. The audio VAE uses hop 882 at 44,100 Hz with patch_size 4, so one step is 80 ms of audio and real time needs 12.5 steps/s. We measured about 8.3 to 8.9 steps/s.
  • The sibling model, Ming-3.0-tts-tiny, is an MoE on Ling V3 (7.9B total, 1.3B active), needs its own environment (Python 3.10.15, torch 2.7.1+cu118), recommends an H800, and is the only one with multi-speaker podcast dialogue. We did not run it.
  • Leaderboards claimed in the GitHub README (Artificial Analysis, October 2026): 1st among open weights on Controlled Voice (Elo 1069, 13th overall), 1st overall on Pronunciation Robustness (93.2%), 3rd among open weights on Provider Voice (Elo 1188, 17th overall). The README does not say whether these are for 0.5b or tiny. We did not check artificialanalysis.ai.

Authors' benchmarks (not reproduced by us)

Authors' benchmark figures shown in our demo video: Seed-TTS English WER 1.58, InstructTTSEval-ZH 72.3, NVBench English 7.60, labelled not reproduced by us

TestMing-3.0-tts-0.5BOthers in the authors' table
Seed-TTS EN WER (lower is better)1.584% (SIM 0.691)Qwen3-TTS-12Hz-1.7B 1.24, CosyVoice3-1.5B 1.45, F5-TTS 1.83, CosyVoice3-0.5B 2.02
Seed-TTS ZH WER0.800%
MiniMax multilingual WERGerman 0.365%, Spanish 1.099% (SIM 0.763), French 4.264% (SIM 0.675)
InstructTTSEval-ZH voice design average72.29%Qwen3-TTS 77.13, Ming tiny 75.37, Mimo-Audio-7B-Instruct 70.50, VoiceSculptor 67.30
NVBench English single CER7.60Ming tiny 5.86, CosyVoice3 7.87

Authors' limitations: speaker similarity is weaker than content accuracy (French SIM is the clearest gap). NVBench Multi trails Single, so stability with several tags per sentence is still improving. Emotion is harder when the words carry no emotional signal. Pitch is less accurate, and open-ended voice design does not yet match the strongest evaluated system. The usage disclaimer forbids impersonating a real person without consent and asks that synthetic speech be labelled as synthetic.

Setup

  • One RTX 2000 Ada 16 GB on our cloud (driver 595.91.07, compute capability 8.9, 70 W limit, AMD EPYC 7352 host with 6 vCPUs), registered as a worker with a local InferenceKey manager. Production was never touched. An RTX A4000 and an RTX 4000 Ada were out of stock.
  • Speed check: one RTX 4090 24 GB (450 W, AMD EPYC 75F3 host), 5 cases.
  • Custom backend ming-3-tts-05b 0.1.0 (7,215-byte package). setup() fetches the Ming source tarball at the pinned commit (skips ming_tiny, figures, docker, cookbooks), runs snapshot_download of the pinned revision (no PDF or figures), builds MingTTS(model_dir, device, text_normalization=True) with the pynini 2.1.6.post1 wheel, applies an optional MING_VRAM_LIMIT_GIB through torch.cuda.set_per_process_memory_fraction, and warms up with the built-in voice US_Emma. bf16, SDPA, authors' defaults, seed 1895.
  • Reference voice: synthetic 5.22 s English clip from our local Pocket TTS stock voice "anna": "Thanks for calling the support line. I can help you reset your password in just a minute." Spanish clone line: "Gracias por llamar. Su nueva contraseña ya está lista; por favor, vuelva a iniciar sesión en todos sus dispositivos."
  • Checks on the pod: openai/whisper-large-v3-turbo with the language forced per case, CAM++ (campplus.onnx, the encoder Ming conditions on) and microsoft/wavlm-base-plus-sv (independent).

Time to ready

PhaseTime
ensure() to ready, empty caches, RTX 2000 Ada60.3 s
of which: worker builds the backend venvabout 32 s
of which: backend setup26.4 s (source 0.9 s, weights 7.1 s, load 2.7 s, warm-up 3.7 s)
Warm restarts (venv and weights cached)13.1, 12.9, 12.7, 13.2 s
ensure() to ready, empty caches, RTX 409047.3 s

Per request, RTX 2000 Ada

Non-streaming POST /endpoint/<project>/ming-tts/v1/chat/completions. GPU s is the backend's synchronised time around synthesize(). Round trip measured on the client, through a tunnel.

CaseTaskAudio sGPU sx real timePeak alloc MiBRound trip sWhisper heard
en-ip-georgeIP UK_George5.38.390.632,9198.52exact
en-ip-emmaIP US_Emma8.212.950.632,92413.18exact ("I am" for "I'm")
en-freeform-coachFREEFORM4.97.790.632,8908.01exact
en-freeform-whisperFREEFORM ("Whisper, tense and frightened")2.94.790.612,8884.96exact
en-design-narratorVOICE_DESIGN4.26.710.632,9266.91exact
en-design-captainVOICE_DESIGN6.410.030.642,92710.24exact
en-design-girlVOICE_DESIGN3.55.660.622,9225.84exact
en-cloneTTS + reference4.57.070.642,9167.53exact
en-clone-tagsTTS + reference, [Laughter] [Sigh] [Uhm]7.011.020.632,92211.49exact + "Um,"
en-clone-notags (control)same text, no tags5.38.050.662,9178.62exact
en-ip-tagsIP, [Breathing] [Cough] [Question-huh]5.58.710.632,9248.92text + "Huh?"
en-ip-notags (control)same text, no tags4.26.400.662,9206.58exact
en-numbersIP US_Emma16.125.040.642,93625.32numbers wrong
en-numbers-words (control)numbers spelled as words5.98.920.662,9209.12"Order Number 40, 520, 1 ... 2020, 6."
en-longIP UK_George, 61 words20.732.410.642,92332.96exact
es-cloneMULTILINGUAL es + English reference6.910.480.662,92010.85exact
es-ip-emmaIP US_Emma + language_id es7.611.600.662,92711.79exact
es-designVOICE_DESIGN + es4.97.640.642,9307.82exact
es-freeformFREEFORM + es5.78.620.662,8948.83exact
es-longMULTILINGUAL es, 52 words18.528.010.662,92228.61exact
es-numbersMULTILINGUAL es11.417.080.672,92717.99numbers wrong
zh-design-authorsVOICE_DESIGN, the authors' Chinese example5.049.840.102,98850.00exact
  • Round trip minus GPU time: 0.13 to 0.9 s (base64 WAV through the tunnel and the message bus, a 20 s clip is about 1.8 MB raw). Across runs, the 20.7 s paragraph took 31.0 to 32.4 s of GPU time.
  • zh-design-authors: the text was split in two fragments and the first ran the whole decode budget (343 steps, 41 s, stop head never fired) before the trailing silence was trimmed.
  • GPU memory: 2,846 MiB allocated after load, 2,888 to 2,936 MiB peak allocated per request (2,988 MiB on the Chinese outlier). nvidia-smi card peak 3,336 MiB, including the CUDA context. GPU utilisation 23 to 31%, about 24 W of 70 W.
  • The backend serves one request at a time (a lock plus workload concurrency 1). No parallel test.
  • With the same seed, outputs are deterministic per card but differ slightly across cards (5.2 vs 5.3 s for en-ip-george, 7.0 vs 6.4 s for the captain).

RTX 4090 speed check

CaseAudio sGPU sx real timeRound trip s
en-ip-george5.26.850.768.44
en-design-captain7.09.050.779.62
en-clone-tags6.98.940.779.80
en-long20.927.11 / 27.790.77 / 0.7527.94 / 28.49
es-clone6.99.230.759.93

nvidia-smi during en-long: 3,562 MiB, 14 to 20% utilisation, about 81 W of 450 W. The step loop is launch and CPU bound. We did not separate how much of the roughly 17% gain comes from the faster host CPU.

Speed on real cards, RTX 4090 0.76 against RTX 2000 Ada 0.65, and the PyTorch memory cap ladder on the 16 GB card: 8, 4 and 3 GiB gave the same audio, 2.75 GiB failed to load, caps emulated

VRAM ladder

The narrative rounds these caps to GB; the exact values are GiB. Caps set with MING_VRAM_LIMIT_GIB (torch.cuda.set_per_process_memory_fraction) on the RTX 2000 Ada, each a fresh ensure() with warm caches. WAVs of en-clone-tags, en-ip-george, en-long and es-clone compared by md5 against the uncapped run.

RungConfigPeak allocSpeedOutputReal or emulated
RTX 4090 24 GBbf16, SDPA, default2,923 MiB (3,562 MiB card)0.75 to 0.77xtext exact on the subsetreal
RTX 2000 Ada 16 GBbf16, SDPA, default (main run)2,936 MiB (3,336 MiB card)0.61 to 0.68xreferencereal
8 GiB capMING_VRAM_LIMIT_GIB=82,923 MiB0.66 to 0.67xbyte-identicalemulated
4 GiB capMING_VRAM_LIMIT_GIB=42,923 MiB0.64 to 0.66xbyte-identicalemulated
3 GiB capMING_VRAM_LIMIT_GIB=32,923 MiB0.65 to 0.68xbyte-identicalemulated
2.75 GiB capMING_VRAM_LIMIT_GIB=2.75setup() failed: CUDA out of memory. Tried to allocate 20.00 MiB ... this process has 2.83 GiB memory in use. 2.75 GiB allowedemulated
RTX 3080 Ti 12 GBtwo attemptsthe machine came up without direct SSH (no port 22 mapping) and the manager terminated it after 180 s, twicereal, never ran

Not tried: any real card under 16 GB, fp16 on pre-Ampere cards, flash-attn, vLLM-Omni, CPU.

Quality details

  • Words: 17 of 19 bench clips word for word. Ladder and 4090 subsets gave identical text.
  • English numbers. The normalizer output in the backend log was Order number four thousand, five hundred twenty，one ships on March three , two thousand twenty，six and costs one thousand, two hundred forty，nine point nine nine dollars , about twelve point five percent less than last year. Call five hundred fifty，five minus one hundred forty，two . The hyphen becomes a full-width comma (a pause), and the phone number 555-0142 is read as a subtraction. Whisper heard "Order number 4520. One ships ... costs $1,240, $9.99 ... Call 550-5-142". Spelling the numbers as words did not avoid it ("forty-five twenty-one" became "40, 520, 1").
  • Spanish numbers skip the normalizer: "1.249,99 euros, un 12,5 %" was heard as "1.1249.99 euros, un 2,5%".
  • Speaker similarity, CAM++ cosine against the reference: en-clone 0.865, en-clone-tags 0.849, es-clone 0.838, es-long 0.853, es-numbers 0.836. Non-clones 0.162 (Chinese) to 0.540 (US_Emma). Designed voices 0.25 to 0.32. WavLM-SV: clones 0.924 to 0.970, but built-in US_Emma (a female voice like the reference) also 0.84 to 0.86, so WavLM separates less. Cross-lingual cloning (English reference, Spanish output) kept the voice as well as same-language cloning by both measures.
  • Tags: 7.0 vs 5.3 s (clone, [Laughter] [Sigh] [Uhm]) and 5.5 vs 4.2 s (built-in voice, [Breathing] [Cough] [Question-huh]). Tag boxes in the key image come from Whisper word timings and are approximate. No human listening test.
  • Voice design and free-form direction in English and Spanish gave exact text. Prompts included "A gravelly old sea captain in his seventies, amused and a little hoarse..." and "Whisper, tense and frightened, very quietly and slowly." Whether the voice matches the description was not measured.
  • No crash, NaN or empty output in about 60 requests.
  • No head-to-head Spanish comparison with our earlier Pocket TTS Spanish fine-tune: no same text, no shared checker, no listening. The only contrast is speed, on different hardware: that fine-tune ran at 1.05x real time on a desktop CPU, Ming at 0.61 to 0.77x on GPU.

Speaker similarity from our demo video: clones 0.87, 0.85 and 0.84 against US_Emma 0.54 and a designed narrator 0.32, and 17 of 19 clips word for word

Step by step with the SDK example

The SDK example is commit b038dd7529651076b5055604584c1fd1b51f517c on inferencekey-sdk main: backend.py, publish.py, main.py, requirements.txt, README.md, .env.example, .gitignore.

You need an SDK token (ik_sdk_), an API key (ik_live_), a project slug and a registered NVIDIA worker (IK_WORKER_ID) with a driver for CUDA 12.4 or newer and about 3 GB of free VRAM. INFERENCEKEY_TENANT_ID and INFERENCEKEY_USER_TOKEN are optional. Without them, upload dist/*.tar.gz from Dashboard, Custom backends, and turn on Chat in the workload's endpoint settings by hand.

cp .env.example .env        # fill in real values
python -m venv .venv && . .venv/bin/activate
pip install maturin requests
( cd ../../bindings/python && maturin develop --release )
python publish.py           # once: packages backend.py and uploads it
python main.py              # ensure -> wait until ready -> 4 WAV files in ./outputs

main.py deploys the workload (ensure(), policy fixed, one replica), turns on the chat route, waits until ready, writes four WAVs and deletes the workload on exit (IK_KEEP_ALIVE=1 keeps it). Our run with IK_KEEP_ALIVE=1:

OutputAudioGPU timex real time
1-built-in-voice (UK_George)5.3 s7.899 s0.67
2-voice-design (narrator)4.2 s6.12 s0.69
3-clone-with-tags ([Laughter], [Sigh])6.0 s9.124 s0.66
4-clone-spanish4.3 s7.201 s0.60

Peak VRAM 2,912 to 2,926 MiB. Ready after 60.3 s.

Request shape. The text is the last user message, a reference goes as an input_audio part, and options go in extra_body (task, language_id, style, instruction, voice, prompt_text, seed, cfg, sigma). The reply's message content is a JSON string with audio_b64 (16-bit WAV, 44.1 kHz), seconds, task and timings (gpu_s, total_s, realtime_factor, peak_vram_mib). Limits: 600 characters of text and 30 s of reference per request.

import base64, json, os, requests

base = os.environ.get("INFERENCEKEY_BASE_URL", "https://cloud.inferencekey.com").rstrip("/")
project = os.environ["INFERENCEKEY_PROJECT"]
api_key = os.environ["INFERENCEKEY_API_KEY"]
slug = os.environ.get("MING_WORKLOAD_SLUG", "ming-tts")
ref_b64 = base64.b64encode(open("reference.wav", "rb").read()).decode()
REFERENCE_TRANSCRIPT = "Thanks for calling the support line. I can help you reset your password in just a minute."

body = {
    "model": "default",
    "messages": [{"role": "user", "content": [
        {"type": "text", "text": "[Laughter] I did not expect that at all. [Sigh] Well, let's try one more time."},
        {"type": "input_audio", "input_audio": {"data": ref_b64, "format": "wav"}},
    ]}],
    "extra_body": {"task": "TTS", "prompt_text": REFERENCE_TRANSCRIPT, "language_id": "en", "seed": 1895},
}
r = requests.post(f"{base}/endpoint/{project}/{slug}/v1/chat/completions",
                  headers={"Authorization": f"Bearer {api_key}"}, json=body, timeout=600)
out = json.loads(r.json()["choices"][0]["message"]["content"])  # IK-531 workaround
open("clone.wav", "wb").write(base64.b64decode(out["audio_b64"]))

For Spanish, German, French, Korean or Japanese use "task": "MULTILINGUAL" with "language_id": "es" (and so on). For a designed voice use "task": "VOICE_DESIGN" with "style". For a built-in voice use "task": "IP" with "voice": "UK_George".

Problems

  1. Slower than real time on both cards (0.61 to 0.77x), with the GPU 14 to 31% busy. The authors' Python step loop sets the pace. vLLM-Omni support is marked done in their roadmap. We did not try that route.
  2. Numbers: English hyphenated number words get a pause and phone numbers become subtractions. Spanish numbers skip the normalizer and come out garbled. Status: upstream behaviour, open. Check numbers in the output.
  3. Chinese voice-design example hit the decode limit once: 49.8 s for 5 s of audio. Not seen in English or Spanish.
  4. /v1/audio/speech cannot carry a custom backend's output: the example uses the chat route and json.loads of the message content. Open, IK-531.
  5. Backend upload refused on our dev machine (XMinioStorageFull, disk about 99% full). Fixed for the run by freeing our own files. A test-environment issue, not one you will see.
  6. RTX 3080 Ti machines without direct SSH (twice): the worker could not be installed and the manager terminated each after 180 s.
  7. Cosmetic: the panel showed an Access "No access" badge while a wildcard key worked.
  8. Upstream load warnings, with correct output: You are using a model of type dense to instantiate a model of type bailingmm, Sliding Window Attention is enabled but not implemented for sdpa, and a transformers past_key_values deprecation.

Remaining samples

Demo video

A 191-second walkthrough: the authors' own demo clips (credited to inclusionAI, Ant Group), their benchmarks, our test as questions and answers, and which GPU. English subtitles.

Ming-3.0-tts deploy test: the terminal shows a cloned line with the lower third '5.6 s of voice in 8.4 s'

Sources

  • Model card: https://huggingface.co/inclusionAI/Ming-3.0-tts-0.5B (HF API fetched 2026-10-28)
  • Code: https://github.com/inclusionAI/Ming-3.0-tts at commit 9332fb94, including requirements.txt, ming_tts.py, modeling_bailingmm.py, fm/flowloss.py, cookbooks/instruction_en.md
  • Authors' benchmarks and leaderboard claims: https://github.com/inclusionAI/Ming-3.0-tts#performance
  • Where we spotted it: AI Search news video of 2026-10-25, chapter at 13:33, https://www.youtube.com/watch?v=EfyQOAJjx1A
  • Our SDK example: https://github.com/inferencekey/inferencekey-sdk/tree/main/examples/ming-3-tts-05b-speech-custom-backend-private-nvidia
  • Our measurements: deploy test of 2026-10-28 (bench, ladder and 4090 logs, per-case summaries, Whisper and speaker-similarity results, nvidia-smi samples)