LFM3-Omni-2B: one 4.5 GB model listens, looks and talks back on 5 GB of GPU

We showed LFM3-Omni-2B a concert flyer, asked out loud when and where the show was, and the GPU had the written answer in under a second: 7:30 PM, Pier 9 Warehouse. A voice assistant usually needs four models for that: one to hear, one to look, one to think and one to speak. This open model from Liquid AI is one 4.5 GB set of weights that does all four. We ran it on an RTX A5000 on our cloud through InferenceKey, and it answered in its own voice too, on about 5 GB of GPU memory.

If you run your own GPUs, that means every stage of a voice assistant fits on one small card. There are two catches. The fast open-source server for this model, SGLang, only writes text, so for speech we used the model's own code, about four times slower at text in our test. The license stops commercial use at 10 million dollars of annual revenue.

LFM3-Omni-2B on the same 9.7 s clip of a toddler reading on a bed, same question. Left, frames plus soundtrack: green boxes on "The child is speaking", which is right, and amber boxes on the quoted words, which we could not verify. Right, frames only: red boxes on "no audible speech or music in the clip", which is wrong

Same clip, same typed question. With the soundtrack on, the model hears a child talking. With frames only, it says the clip is silent. The run with sound also carried a one-line instruction to use what it sees and hears, so two settings changed, not one. Still, only the run that got the sound could tell it was there.

Three tiles: about 5 GB of GPU memory, a 10-second spoken answer in about 7 seconds of GPU time, and a 16 GB card as fast as a 24 GB one

One small model where a voice assistant usually chains four

Liquid AI's model card says it "replaces a separate ASR model, vision-language model, LLM and TTS model". In plain words: speech recognition, a vision model, a language model and a speech synthesizer, in one network. Our test asked whether that holds up on one small card.

It takes text, images, video with its soundtrack and recorded speech. It answers in text, in speech, or in both at once. In the last mode it writes a sentence and speaks it in the same pass, so a screen and a speaker get the same answer. By Liquid AI's list it speaks 19 language and accent combinations, from Arabic to Chinese, with female and male voices; we tried English and Spanish. It can keep about 32,000 word pieces in view: a long written transcript, but only a few minutes of video, since our one-minute clip alone took about 8,500. It answers at once, with no hidden reasoning step first.

Liquid AI built it on LFM3-1.7B, their new small language model. They added an image reader, a sound reader that starts while you are still speaking, and a part that turns the output into audio. Liquid AI's LFM3 announcement is dated 27 October. It is one of six small models in the family. We spotted it in AI Search's news video, whose narrator summed it up as one tiny model that can "see, hear, understand, and even talk back to you".

The authors compare it with other small open models that also take sound and video. These are their numbers. We did not reproduce them.

Test (authors' numbers)	LFM3-Omni-2B	Qwen2.5-Omni-3B	Qwen2.5-Omni-7B
Daily-Omni, questions about everyday video with sound	70.09	58.81	61.32
OmniBench, mixed image and audio reasoning	46.32	47.11	49.65
English transcription error rate (lower is better)	6.36	6.30	not listed

It wins the first row by a wide margin and loses the second to both Qwen models. On transcription it is level with the smaller Qwen. The authors print the losses too, which makes the table easier to trust. The full set, including Google's Gemma 4 small models, is in the appendix.

Our test: two small cards and a handful of short recordings

We ran the full, uncompressed model on two GPUs on our cloud through InferenceKey: an RTX A5000 with 24 GB for the main run, and an RTX 2000 Ada with 16 GB on a fresh machine. Both used the same code.

The cases were small and practical. A spoken question about a mountain, answered by voice. A made-up concert flyer plus a spoken question about it. A clip from a public audiobook dataset to transcribe. A spoken request to turn the volume down, which should come back as a command an app can run. Two sentences to read aloud, in English and in Spanish. And a 10-second clip from Liquid AI's own examples, with and without its sound.

The limits, once. One request at a time. Our own short inputs, not the authors' benchmarks. No human judged the voice: a separate speech recognizer, Whisper, read the audio back. We did not test the compressed builds, the realtime voice server Liquid AI points to, or any real card under 16 GB. We also ran no chain of four separate small models on the same card, so nothing here shows the one-model route is faster, smaller or better than that pipeline.

A 10-second spoken answer: made in about 7 seconds, delivered in 8 to 10

A 10-second spoken answer was ready on the GPU in about 6 to 7.5 seconds. It reached the caller after 8 to 10 seconds, because our setup sends the audio as one file at the end. Every spoken answer took less time to make than to play: a 4-second sentence took 2 to 3 seconds.

What we asked	What came back	GPU time (A5000 / 16 GB card)
A spoken question about Africa's tallest mountain, answer by voice	Kilimanjaro, 5,895 meters, about 10 s of speech	7.4 s / 6.6 s
Concert flyer plus a spoken question on when and where	"The concert starts at 7:30 PM, and it is located at Pier 9 Warehouse, 410 Bay Street."	0.84 s / 0.93 s
12.5 s audiobook clip, transcribe	Exact, apart from capitals and commas	1.16 s / 1.01 s
A spoken request to set the volume to 30, with a volume tool	`set_volume({"volume": 30})`	0.56 s / 0.45 s
Read a Spanish sentence aloud	Whisper heard "El museo abre a las 10 de la mañana y cierra a las 6 de la tarde."	2.3 s / 2.2 s

This is the model's own voice answering the flyer question:

The model's spoken answer to the flyer question, 7.4 s, generated on the A5000

Inside the server, the first piece of speech was ready 0.2 to 0.5 seconds in, so streaming could cut the wait, but we did not build or test streaming.

Where it slips: a number in feet, and words in a toddler's babble

The plain facts were right. The slips were a unit conversion and two descriptions of video. We asked for the height of Kilimanjaro by voice ten times, on both cards and with several memory limits. The meters were right nine times out of ten, and the feet were wrong in most runs. Spoken answers vary from run to run, because the voice is generated with some built-in randomness and the words follow it. A voice assistant that reads out numbers needs a check, or a tool, for conversions.

The video answer is the other caution. The model quoted the child as saying "I want to look at the book". That sentence is unverified. Whisper heard "I don't want to put it in my bag". The model itself, asked only to transcribe the sound, heard "I want, I want to put them in". The child is hard to understand, so treat quoted words from unclear audio as a guess.

On a one-minute clip of Liquid AI's own street demo, it described the scene and read the shop signs right. It also called real street footage "a driving video game".

Which GPU: about 5 GB of memory, and a real 16 GB card ran it as fast as 24 GB

The model needs about 5 GB free for voice, photos and short clips, and about 6 GB for a one-minute video. The smallest real card we ran is the RTX 2000 Ada with 16 GB, a 70-watt workstation card. It gave the same results as the 24 GB A5000 with no slowdown.

To see how low it goes, we capped the memory the model's own code could use on the A5000, step by step. That is a software limit on a bigger card, not a smaller card, and the whole process needs about 0.4 GB on top of the cap.

Card or memory cap	Voice, photo, transcription, speech and a 10 s video
RTX 2000 Ada 16 GB, real card	all pass, real card
Model memory capped at 4.8 GB (simulated)	all pass
Model memory capped at 4.7 GB (simulated)	the 10 s video runs out of memory

The serving split: the fast server is four times quicker at text, and cannot speak

With an unreleased fix, the fast server, SGLang, writes text four times quicker, but it cannot speak.

This is the problem you will hit first. Liquid AI's docs give a one-line SGLang command, but the released SGLang (version 0.5.23) has no support for this model, and on our card it crashed during start-up. The fix exists as a proposed change, #46858, still open when we checked on the day of the test. InferenceKey's ready-made SGLang server uses the released version, so it cannot serve LFM3 until that fix ships. Status: waiting on SGLang.

We built that change by hand, outside the platform, to see what it is worth. With identical answers, a paragraph took about one second instead of four (158 word pieces per second against 38 to 40). For one caller at a time, the slower route is fine; we did not test several at once. For heavy text traffic, wait for the release.

The license: commercial use stops at 10 million dollars of revenue

The weights come under the LFM Open License v1.0, not a standard open-source license. Commercial use is allowed only while your organization's annual revenue stays under 10 million US dollars. Above that, commercial use is not licensed. Qualified non-profits are exempt for research and non-commercial use.

For a startup or a small product team, that is open enough to ship. For a large company, it means talking to Liquid AI about other terms before the first deployment, not after. The license text is short.

On InferenceKey: ready in under a minute, one custom backend for every mode

One custom backend on one GPU on our cloud served voice, photos, video and commands from one address, ready in under a minute. A custom backend is a short Python file you upload to InferenceKey: the platform installs its dependencies on your GPU machine and sends requests to it. None of InferenceKey's built-in engines can make this model speak (llama.cpp's realtime server can, but we did not test it), so ours drives the model's own code and returns the text and the audio file in one answer.

On the A5000 the model was ready 46 seconds after we asked for it. On the fresh 16 GB machine, which had to install everything from scratch, it took 59.5 seconds. The 4.5 GB download itself took 5 to 6 seconds.

The platform ran from a local copy of InferenceKey, same code as the hosted service. The SDK example pins the model to one GPU machine you register, so the model runs on hardware you control. Requests still pass through the InferenceKey API unless you host InferenceKey itself on your own servers.

Should you use it? Yes to build a voice assistant on your own small card, with a number check

You want a voice assistant that hears, sees and talks on your own hardware: yes, to build on, with a check for numbers it reads out. As tested, the caller waits for the whole answer.
You need fast text for heavy traffic: not yet. Wait for the SGLang release.
You have a card under 16 GB: the memory caps above say about 5 GB free is enough.
Your company makes 10 million dollars a year or more: talk to Liquid AI before you build on it.

To hear three of the same answers from your own GPU (the voice answer, the flyer and text to speech), run the LFM3-Omni-2B SDK example. It needs a project, an SDK token, an API key and an NVIDIA GPU machine you register (a recent card) with about 5 GB free. Upload the backend once with python publish.py, then run python main.py. It deletes the deployment when it ends.

For engineers

Credits

Liquid AI: the model, the LFM3-1.7B backbone, the vision encoder, the causal audio encoder, the LiquidCodec audio tokenizer, the model card, the benchmarks, the docs, the GGUF builds and the sample videos (including the toddler clip from their llama.cpp sample and the street demo from their launch video).
The SGLang PR author (tugot17, sgl-project/sglang#46858): the SGLang implementation we built by hand.
The Hugging Face Transformers maintainers: the library the custom backend runs on (trust_remote_code, AutoModelForMultimodalLM).
The llama.cpp maintainers: GGUF and realtime server support (not tested by us).
OpenAI Whisper large-v3 (via faster-whisper): the independent read-back of every WAV. LibriSpeech: the transcription clip.
Kyutai Pocket TTS ("alba" voice): generated our spoken test questions.
AI Search: where we spotted the model (2026-11-01 news video, LFM3 chapter at 44:17).
Our part: the custom backend, the deployment through the platform on two cards, the test inputs, the soundtrack check, the memory ladder, the SGLang attempts, the SDK example and the demo video.

Model and files

LiquidAI/LFM3-Omni-2B, revision 21d75123f86c1f7ffb4bef1ea202710c4e7ebe67 (created 2026-10-21, last modified 2026-11-03, not gated). 4,499,839,711 bytes: model.safetensors 4,333,941,640 B, audio_tokenizer/model.safetensors 147,143,540 B, tokenizer.json 17,908,634 B. Some catalogues list "5.4 GB": that is a VRAM estimate, not the checkpoint.
2.17B parameters. Inputs: text, images and video at native resolution (each 2x2 patch block is one token, up to 2,048 tokens per image), video at 2 fps with 4 to 384 frames by default, 16 kHz audio. Outputs: text, 24 kHz speech, or interleaved. Context 32K tokens (input plus output). 128,000-token text vocabulary.
Speech locales (19): Arabic, Danish, German, Greek, British English, American English, Finnish, French, Hebrew, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish, Swedish, Turkish, Chinese. Text languages (16): English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish.
Non-reasoning: enable_thinking=True is not supported, every answer opens with an empty think block.
Requires transformers>=5.20.2 with trust_remote_code. Video with sound needs load_audio_from_video=True and use_audio_in_video=True, otherwise frames only.
Recommended sampling: greedy for text; for speech audio_temperature=0.4, audio_repetition_penalty=1.2 (docs add audio_repetition_window=64).
GGUF repo LiquidAI/LFM3-Omni-2B-GGUF (sha d508b88f, created 2026-10-27): F16/BF16 3.47 GB, Q8_0 1.85 GB, Q4_K_M 1.10 GB, Q4_0 1.05 GB; mmproj F16/BF16 0.95 GB, Q8_0 0.51 GB, Q4_K_M 0.29 GB, Q4_0 0.27 GB. Needs llama.cpp b11960 (v0.8.0) or newer; realtime voice with tool calls on the /v1/realtime WebSocket of llama serve, for example llama serve -hf LiquidAI/LFM3-Omni-2B-GGUF:Q4_K_M --reasoning off --temp 0 --port 8080; speech synthesis via llama-tts. Not tested by us.
Liquid AI blog: six ~2B LFM3 models from LFM3-1.7B-Base, 22T tokens of pre-training. The causal audio encoder can prefill while the user speaks ("as low as 50 ms" to first token); LiquidCodec decodes at about 87x real time on one CPU thread (Apple M4 Max), 12 tokens per 80 ms of speech. Their claims, not measured by us.

Authors' claims (not reproduced)

All measured by Liquid AI, from the model card:

Benchmark	LFM3-Omni-2B	Qwen2.5-Omni-3B	Qwen2.5-Omni-7B	Gemma 4 E2B	Gemma 4 E4B	Phi-4-mm
Daily-Omni	70.09	58.81	61.32	50.63	54.14	48.87
WorldSense	47.10	44.99	45.37	38.11	43.51	42.37
OmniBench overall	46.32	47.11	49.65	16.81	20.67	38.97
English ASR WER (lower is better)	6.36	6.30	not listed	8.26	7.82	5.14
English TTS UTMOSv2	3.80	3.42	not listed	not listed	not listed	not listed
VoiceBench	64.73	66.37	not listed	63.28	72.90	64.97
Tool calling (internal eval)	52.20	44.30	not listed	55.60	72.40	not listed

"Not listed": the card's audio table has no Qwen2.5-Omni-7B column, gives UTMOSv2 only for LFM3-Omni-2B and Qwen2.5-Omni-3B, and has no tool-calling score for Phi-4-mm. On tool calling, LFM3-Omni-2B beats Qwen2.5-Omni-3B and trails both Gemma 4 models. The card also reports AV-SpeakerBench, OmniBench speech, spoken visual QA, vision, multilingual ASR and MMAR results, not copied here.

Setup

GPUs on InferenceKey Cloud: RTX A5000 24 GB (24,564 MiB, 230 W, compute capability 8.6, driver 595.91.07, CUDA 13.2) for the main run; RTX 2000 Ada Generation 16 GB (16,380 MiB, 70 W, compute capability 8.9) for the real 16 GB rung, on a fresh worker. The A5000 was the first 24 GB card that came up; the RTX 2000 Ada was the smallest real card available.
Other route on the card: GGUF builds for llama.cpp, on CPUs and GPUs, with a realtime voice server (details under Model and files). Not tested.
Backend: a custom Python backend (lfm3-omni-2b 0.1.0, task text2audio, chat route) on transformers 5.22.2, torch 2.15.0, torchvision 0.29.2, BF16. Why custom: speech and interleaved output exist only in the checkpoint's Transformers code; no InferenceKey built-in engine serves speech for this model (llama.cpp's realtime server is not tested here), and released SGLang does not serve it at all.
setup() installs ffmpeg if missing, downloads the checkpoint at the pinned revision, loads AutoModelForMultimodalLM.from_pretrained(dtype=bf16, trust_remote_code=True), the processor and LiquidCodec, and warms up text and speech. process() takes OpenAI messages (text, input_audio, image_url / video_url as data URLs) and returns text, a base64 24 kHz WAV, parsed tool_calls and timings (first text token, first speech frame via streamers, GPU seconds, peak VRAM).
A local InferenceKey manager (same code as the hosted service). Production was never touched.
Greedy text; speech with audio_temperature 0.4 and audio_repetition_penalty 1.2. Spoken inputs made with Kyutai's Pocket TTS ("alba" voice), run locally; the flyer is a made-up image. The video questions were typed, and every video answer was text only (mode text).

Step by step with the SDK example

Clone inferencekey-sdk and open examples/lfm3-omni-2b-voice-vision-custom-backend-private-nvidia (commit ba49628ec5b24b68e8c404b1e54772df6b499f6b).
cp .env.example .env and fill in INFERENCEKEY_SDK_TOKEN, INFERENCEKEY_API_KEY, INFERENCEKEY_PROJECT and IK_WORKER_ID (a registered NVIDIA worker, bf16-capable Ampere or newer, about 5 GB of free GPU memory, 6 GB for a one-minute video, a driver for CUDA 12.8 or newer, ffmpeg for video). For publish.py also INFERENCEKEY_TENANT_ID and INFERENCEKEY_USER_TOKEN.
Install the client and the local SDK build:

python -m venv .venv && . .venv/bin/activate
pip install requests
pip install maturin && ( cd ../../bindings/python && maturin develop --release )

python publish.py once: it packages backend.py and uploads it to your tenant (or upload dist/*.tar.gz in Dashboard, Custom backends, and turn on "Chat" in Endpoint settings).
python main.py: ensure() declares the workload (backend="lfm3-omni-2b", task_type="text2audio", fixed policy, one replica) pinned to your worker, waits until ready, sends three requests (spoken question to interleaved speech, flyer plus spoken question, text to speech), writes answers to ./outputs and calls delete() unless IK_KEEP_ALIVE=1.

Request options in extra_body: mode (text, audio, interleaved), max_new_tokens (one 80 ms speech frame counts as one), temperature, audio_temperature, audio_repetition_penalty, use_audio_in_video (default true), fps, tools (OpenAI function schemas). LFM3_VRAM_LIMIT_GIB caps PyTorch memory. Send "model": "default". Inline media count against a request limit of about 2 MiB of JSON: a 480p, 10 s MP4 is about 230 KB.

Timings

Step	RTX A5000	RTX 2000 Ada
`ensure()` to ready	46.0 s (warm uv cache)	59.5 s (fresh worker, empty uv cache)
venv build plus setup	20.0 s
Weights download (pre-fetched in parallel)	5 s	6 s
Load	3.6 s
Warm-up (text and speech)	3.1 s
Allocated after load	4,274 MiB

Full measurements (through the platform)

Case	RTX A5000	RTX 2000 Ada 16 GB	Output
Spoken question (3.3 s) to interleaved text plus speech	10.48 s of speech in 7.418 s (1.41x), round trip 10.11 s; test run 9.76 s in 6.938 s, first text 0.047 s, first speech frame 0.275 s	10.08 s in 6.563 s (1.54x), round trip 8.24 s; test run 10.08 s in 6.148 s (1.64x), first frame 0.203 s	Kilimanjaro, 5,895 m right; feet wrong (19,004 / 19,34 / 19,031)
Flyer image plus spoken question, text	0.836 s, 1,135 prompt tokens	0.931 s	right
Same, interleaved speech	7.12 s of speech in 5.41 s (1.32x), first frame 0.35 s	6.32 s in 4.26 s (1.48x)	same facts, spoken
ASR, LibriSpeech 1272-128104-0002 (12.5 s)	1.16 s	1.01 s	exact apart from case and punctuation
9.7 s video with soundtrack, typed question, system prompt "Answer questions about the video using what you see and hear."	2.63 s	2.47 s	"The child is speaking, saying, 'I want to look at the book'" (quote unverified)
Same video and question, frames only, no system prompt	5.38 s	4.87 s	"there is no audible speech or music in the clip" (wrong)
Spoken tool request with `set_volume` schema	0.56 s	0.45 s	`set_volume({"volume": 30})`
TTS English female / Spanish male	4.64 s in 2.93 s (1.58x) / 3.68 s in 2.26 s (1.63x), first frame about 0.05 s	4.8 s in 2.68 s (1.79x) / 4.16 s in 2.19 s (1.9x)	Whisper reads back the sentences
Text, 146-token paragraph	37.6 tok/s	40.5 tok/s	right
60 s video, 8,472 prompt tokens	5.9 to 6.4 s, 5,966 MiB process	5,824 MiB process	scene and signs right, real street called "a driving video game"

The two video runs differ in two settings (use_audio_in_video and the system prompt), so their prompt sizes and times are not a clean audio-only comparison; a frames-only rerun with the same system prompt was not done. First-speech-frame times are measured inside the backend from generation start. The backend returns the whole answer as one base64 WAV, so the client receives audio only at the end of the round trip.

GPU memory: PyTorch peak 4,285 to 4,434 MiB per request, process 4,772 to 4,832 MiB in nvidia-smi. The soundtrack has four bursts above -35 dB (silencedetect). Whisper large-v3 with auto language heard "Ola, ola..."; forced English "I don't want to put it in my bag."; LFM3 audio-only ASR "I want, I want to put them in."

VRAM ladder

Emulated rungs use torch.cuda.set_per_process_memory_fraction on the A5000. Five cases: ASR, image plus voice, TTS, voice to speech, 10 s video. The cap limits PyTorch memory only, not the whole process: the nvidia-smi process figure adds about 400 MiB of non-PyTorch memory (4,832 MiB process against a 4,434 MiB PyTorch peak). In the narrative, 4.45 GiB is 4.8 GB and 4.35 GiB is 4.7 GB; 4,772 to 4,832 MiB is 5.0 to 5.1 GB.

GPU or cap	Config	Peak memory (MiB, nvidia-smi process unless noted)	Time per output	Output OK	Real or emulated
RTX A5000 24 GB	BF16, custom Transformers backend, through the platform	4,832 (4,434 PyTorch)	7.4 s (voice answer)	yes	real
RTX 2000 Ada 16 GB	same, fresh worker through the platform	4,434 PyTorch peak; 5,824 process for the 60 s video	6.6 s	yes, same answers	real
8 GiB cap	5 cases	4,832	6.9 s	yes	emulated
6 GiB cap	5 cases	4,832	5.1 s	yes	emulated
5 GiB cap	5 cases	4,832	6.8 s	yes	emulated
4.6 GiB cap	5 cases	4,832	7.2 s	yes	emulated
4.45 GiB cap	5 cases	4,772	6.8 s	yes	emulated
4.35 GiB cap	5 cases	4,718		no: 10 s video `OutOfMemoryError ... Tried to allocate 20.00 MiB ... 4.35 GiB allowed`	emulated
5.6 GiB cap, 60 s video	8,472 prompt tokens	5,966 (5,242 PyTorch)	5.9 s	yes	emulated
5.2 GiB cap, 60 s video	8,472 prompt tokens			no: `OutOfMemoryError ... Tried to allocate 142.00 MiB ... 5.20 GiB allowed` (also OOM at 4.6)	emulated

Speeds were unchanged under the caps. The voice-to-speech answers were not identical across caps: Kilimanjaro came back as 5,895 m with 19,031, 19,341, "19,34", 19,285 and 19,000 feet, and once (6 GiB cap) as "five thousand seven hundred and three meters". Not tried: real 6, 8 or 12 GB cards (none smaller than 16 GB was available on our cloud for this test), the GGUF builds and the llama.cpp realtime server, quantized weights, concurrency, CPU.

SGLang

Built-in sglang backend through the platform (PyPI sglang 0.5.23, sglang serve ... --mem-fraction-static 0.35): first refused by the worker's VRAM preflight, "GPU out of memory before sglang spawn: only 19328 MiB free" (it wants 80 % of the card free regardless of --mem-fraction-static). With INFERENCEKEY_VRAM_MIN_FREE_MB=9000: "Lfm3OmniForConditionalGeneration has no SGLang implementation, falling back to Transformers implementation", then "Exception: Capture cuda graph failed: The size of tensor a (48) must match the size of tensor b (16) at non-singleton dimension 2". Not served.
PR #46858 built by hand on the A5000, outside the platform (sglang 0.5.6.post3.dev14179+g47737555a, SGLANG_BUILD_RUST_EXTS=none, --mem-fraction-static 0.3): ready 185 s cold (CUDA graphs) / 47 s warm, process 7,036 to 7,338 MiB. Text 157 to 159 tok/s on a 137-token paragraph (4.2x Transformers). Image plus spoken question 0.41 to 0.43 s. Video with soundtrack 0.95 s warm, but only with a top-level use_audio_in_video: true (mm_processor_kwargs is ignored and gives the frames-only answer). All greedy answers identical to the Transformers backend. Text output only. input_audio with format: flac returned 400.
Liquid AI's docs command: python -m sglang.launch_server --model-path LiquidAI/LFM3-Omni-2B (text output). It needs that PR until a release includes it. PR status and the PyPI version were checked on 2026-11-07 at 06:40 UTC.

Remaining problems

SGLang support unreleased (above). Our built-in backend installs PyPI sglang, so it cannot serve LFM3 yet. Open, upstream.
Worker VRAM preflight blocks a second engine on a card where the first one already runs (80 % free rule). Workaround: INFERENCEKEY_VRAM_MIN_FREE_MB. Open on our side.
False ready: wait_until_ready for the sglang workload printed "ready [server_up]" after 23 s while uv was still installing sglang; another workload on the same worker was the one up. Open on our side.
Endpoint model name: "model": "lfm3" returns 400 model_not_available; send "model": "default". Documented in the example.
No streaming of speech to the client in this backend: the client waits for the whole WAV. Not built.
Transformers text decode is slow: 38 to 40 tok/s, the same on the 70 W card as on the A5000. Transformers also logs that causal_conv1d_fn falls back to its reference PyTorch implementation because causal_conv1d is not installed ("This is correct but much slower"); whether that explains the speed is unverified.
Model slips: Kilimanjaro in ten voice runs: meters right nine times (5,703 once), feet right once (19,341) and off in the other eight that gave feet; interleaved answers vary with audio sampling; quoted words from unclear audio unverified; real street footage called "a driving video game". Whisper read one Kilimanjaro WAV as "19,034 feet" while the text said "19,34", so audio and text may differ slightly on that number.
Two machines on our cloud (an A5000 and an A4000) never opened direct SSH within 180 s and the manager discarded them before the run.

Demo video

The 172-second narrated explainer: Liquid AI's own examples with credit, an architecture insert and a benchmark insert from the model card, then our deployment on the A5000, the model's own spoken flyer answer played without narration, the spoken tool call, the key comparison and the memory ladder. Subtitles available.

More samples

Sources

LiquidAI/LFM3-Omni-2B (model card, configs, modeling code).
LFM Open License v1.0.
LiquidAI/LFM3-Omni-2B-GGUF.
Introducing LFM3: Every Modality for Specialized Intelligence (Liquid AI blog, 27 Oct 2026).
Liquid AI docs, SGLang deployment.
sgl-project/sglang#46858.
AI Search news video, 2026-11-01 (where we spotted it).
SDK example, commit ba49628ec5b24b68e8c404b1e54772df6b499f6b.
Earlier single-stage tests on this site: Ming TTS, Nemotron Transcribe, Underdog Bark.