Hy-Nomas on one RTX 4090: a video world that remembers, about 90 seconds per 5-second turn

Most open video models hand you one clip and forget it. Hy-Nomas, an open model from Tencent Hunyuan, keeps the world in memory, so each new instruction adds five more seconds to the same scene. The question for anyone with their own hardware is whether Tencent's one-card recipe works away from its eight-GPU home. On a single 24 GB RTX 4090 on our cloud, it ran, at about a minute and a half per 5-second piece. The fox, the stone path and the light carried over from call to call. One scene change in four landed: new flowers yes, snow and a storm no. So one rented gaming card can prototype scenes that stay consistent turn to turn, not run a live game. The three-turn fox world is packaged as an SDK example you can replay on your own card.

What you want to know	What we measured
Time for each 5 s of video	88 to 94 s on one RTX 4090
GPU memory	18.5 GiB peak, about four fifths of the card
Smallest setup that worked	a 24 GB RTX 4090, and the same card held to 20 GiB for two turns
Does the world hold together?	yes, over 3 turns (15 s of video)
Does it follow changes?	1 of 4 scene changes (a new object yes, weather no), actions ignored
What you download	about 35 GB of a 196 GB repository

A model that keeps notes on the world it already drew

Hy-Nomas makes video in turns, and each turn starts from what came before. A usual video model is a painter who gets one brief, paints, and leaves. Hy-Nomas is a painter who keeps a notebook. Every time it finishes five seconds, it writes down what the scene looks like. When you send the next instruction, it reads the notebook first. That is why the fox in turn three is the same fox as in turn one.

The authors call it a video world model: you steer it turn by turn, each 5-second segment with a new text instruction or a camera path. The fast version makes 720p video at 24 frames per second, according to the model card. The paper, "Hy-Nomas: Towards Free-form Interactive World Generation", went up on 27 October with 39 authors. It reads your instructions with a borrowed open model, Qwen2.5-VL-7B.

Weights and code are Apache-2.0: free for commercial use, no form to fill in. The code repository lists a few third-party parts under their own licenses.

The download looks scarier than it is. The whole Hugging Face repository is 196 GB, because it carries full-size originals, two add-on adapters (cinematic look, robotics) and a large prompt rewriter. The one-card setup needs only three parts, two of them compressed: the video model, the decoder that turns it into frames, and the text reader. That is about 35 GB, and our machine pulled it in about 20 seconds.

The authors run it on eight GPUs, and publish a second recipe for one

The default launch on the model card spreads one video across eight GPUs that work as a group. Next to it, the authors publish a recipe for a single RTX 4090. It stores the big parts in a compressed form and keeps them in the computer's main memory. Each part moves to the GPU only for its own stage: the text reader, then the video model, then the decoder that turns the result into frames. Only the most recent few seconds of the notebook stay on the GPU. Older pages go to main memory.

The model card is honest about the trade-off. It warns that main memory must hold the parked models and the growing archive, and that the moves between memory and GPU add time. The paper gives no speed for the 4090. That was the question we brought: does the one-card recipe work outside Tencent's setup, and how slow is it?

The model card adds that the one-GPU claim covers video, not the prompt rewriter that polishes each instruction. Their demo rewrites every instruction with Nomas-PE, a much larger model whose provided launcher uses a whole 8-GPU machine by default.

Our test: one RTX 4090, the authors' recipe, three turns per world

The limits first. We built two short worlds, the longest three turns, so 15 seconds of video. We wrote the prompts by hand, without the authors' rewriter, and used one fixed random number (a seed) per world. We did not test camera paths. That checks whether the recipe runs and holds a scene together, not how far a world can go.

We used the authors' own one-card recipe unchanged, on one RTX 4090 on our cloud. Every request went through our platform's API, the way an application would call it. The card was on our cloud. Our platform software ran on our own server, so the hosted service itself is untested here.

The first world is the authors' own example: a small orange fox walking along a stone path in a forest. In turn two we asked for blue flowers to open beside the path. In turn three we asked for snow, and for the fox to stop and look up. A second world, a lighthouse keeper with a lantern, tried a storm.

About 90 seconds per 5-second segment, and the world held together

On one RTX 4090, each 5-second segment took 88 to 94 seconds once the program was warm. That is about 18 times slower than real time. If that pace holds, a one-minute world is twelve turns and about 18 minutes of GPU work. The first call after a fresh start took up to 138 seconds.

Fox world, one RTX 4090	GPU time	Peak GPU memory
Turn 1: the fox walks the path	88.4 s	17.4 GiB
Turn 2: blue flowers open	93.1 s	17.6 GiB
Turn 3: snow, fox stops	88.2 s	17.9 GiB

Memory rose a little with each turn, for a reason we did not find. Counting everything on the card, the peak was 18.5 GiB, about four fifths of the 24 GB card. Of each 90 seconds, about 35 go to drawing, about 9 to writing the notebook, and the rest to turning it into frames and an MP4 file.

The world held together, and that is the part that works best. The fox kept its fur, its size and its walk. The path, the trees and the light stayed put. In the keeper's world, the raincoat, the lantern and the lighthouse carried over between calls. Each call is a separate request, a minute and a half apart, and the scene still read as one take.

It is also repeatable. The same turns with the same seed gave byte-identical video files across four runs. For a team, that means a bug report can include the exact inputs, and someone else on the same setup gets the exact same clip. We checked this on one machine only.

One scene change in four landed: a new object yes, weather and actions no

Hy-Nomas added the object we asked for, on cue. In turn two, blue flowers appeared beside the path as the fox walked past. They did not glow, as the prompt said, but they were there, in the right place.

Changes to the whole scene never appeared. Turn three asked for snow, and the sky stayed sunny. We retried with stronger wording, and still no snow. In the keeper's world, a storm with rain and lightning did not come either. That makes one of four scene changes. The actions failed too: the fox kept walking instead of stopping to look up, and the keeper never raised his lantern. Even the opening prompts landed only in part. The forest had no fireflies and no glow, just daylight and mist.

It played like a band that holds the groove for three songs and misses the key change the singer called. One suspect is the missing rewriter. The authors' demo turns every short instruction into a structured generation prompt before the model sees it. We have not tested whether Nomas-PE fixes snow and storms, so treat that as a guess. The paper adds its own caveat: the one-card shortcuts "may introduce subtle differences that become more noticeable over successive segments".

Which GPU: a real 24 GB RTX 4090 worked, and a 20 GiB limit on it passed too

A 24 GB card works, a 20 GiB limit works, and 16 GiB fails. The RTX 4090 was the only real card we tested. For the smaller sizes, we told the software to use less of that same card.

RTX 4090, 24 GB, real card: works (see above).
Limited to 20 GiB on the same card: works, with no slowdown (90.5 seconds for the second turn) and byte-identical video, over two turns.
Limited to 16 GiB: fails before any video is drawn. It runs out of memory while moving the text reader onto the GPU. A lower resolution would not help, since the failure comes first.

Two caveats change the shopping list. First, part of the recipe only runs on NVIDIA's Ada generation, the family of the RTX 4090. A newer RTX 5090 or an older A40 should fail it, but we did not run them. Second, the authors' own one-card run peaked at 20.53 GiB, with longer segments than ours. So a real 20 GB card is a gamble we did not test.

Main memory matters too. Each 5 seconds of world adds about 5.9 GiB of old notebook pages there. Our example asks for 64 GB, a choice we made, not a measured minimum.

On InferenceKey: one call per turn, the world kept between calls

An interactive model needs a server that keeps the world in memory between calls. Our platform kept it with a small custom program, which only wraps the authors' generation service and one-card recipe. It holds one world in GPU and main memory between calls, so one card serves one world, and one user, at a time. Each call sends a turn and gets back an MP4 of five seconds. The next call continues the same world. With the weights already on disk, the deployment was ready 19 to 31 seconds after the deploy call.

The one problem you will hit if you start from the authors' own scripts is memory. Their live-demo script turns on a mode that decodes the video while the video model is still on the GPU. On 24 GB, that runs out of memory. We turned the mode off, so decoding starts after drawing ends. Status: worked around in our example (that mode off), not yet reported to the authors. The other problems we found, and their fixes, are in the appendix.

Should you use it? Yes for scenes that stay consistent on one card, not yet for live play

If you want a scene that stays consistent from turn to turn, try it.
If you need it to follow scene-wide changes, like weather or a sudden action, test your own prompts first. Ours mostly failed.
If you need real time, look elsewhere. Five seconds of video take a minute and a half on a 4090.
If your card is not an Ada card with 24 GB, expect to rework the recipe. Plan main memory too, and more of it for longer worlds.

To play the same three-turn fox world on your own card, run the Hy-Nomas world model SDK example. It needs an InferenceKey project with an SDK token and an API key. It also needs a private NVIDIA machine with an Ada 24 GB card such as an RTX 4090, 64 GB or more of main memory and about 60 GB of disk. The steps are in the appendix.

For engineers

Credits

Hy-Nomas: Tencent Hunyuan, paper by 39 authors, arXiv 2610.36729. They trained the model and wrote the single-GPU low-memory preset, the INT8 and FP8 weights and the resident generation service our backend wraps, all under Apache-2.0.
Diffusers conversion: published by Tencent as Hy-Nomas-Diffusers; it depends on a Diffusers fork by jiaxiangc (github.com/jiaxiangc/diffusers@hy-nomas).
Text encoder: the language part of Qwen2.5-VL-7B-Instruct, from the Qwen team (Apache-2.0).
Decoder: fine-tuned by Tencent from PixelUMM (Wei et al., 2026), cited in tech report 6.2.
FP8 decoder kernel: built on NVIDIA's CUTLASS (Ada blockwise GEMM example).
Demo video clips: the authors' promo footage, credited on screen as "Source: Tencent Hunyuan, Hy-Nomas project page (promo video)".
Where we spotted it: AI Search's news video of 2026-10-25, Hy-Nomas chapter from 9:27 to 11:14.
Our part: the custom backend around the authors' service, the deployment through the platform, the prompts, the measurements, the VRAM ladder, the SDK example and the demo video.

Model and files

tencent/Hy-Nomas, Apache-2.0, not gated, last modified 2026-10-28. Code: Tencent-Hunyuan/Hy-Nomas main effb8a49b83e6f1d35e26710db35194f2fd952a7 (2026-10-30). GitHub's API shows the license as "Other/NOASSERTION" because the LICENSE file wraps Apache-2.0 with a list of third-party components.
Architecture (model card): dense MMDiT, 54 double-stream blocks, 0 single-stream, hidden 2048, 16 heads x 128, FFN 8192, 3D RoPE, flow-velocity prediction with Euler ODE sampling. The arXiv abstract calls it an "8B diffusion backbone". Text encoder: Qwen2.5-VL-7B language model (3584-dim, two-layer token refiner). Latents: DCAE-V f32t4c32. Decoder: fine-tuned PixelUMM, 36 layers, 4096 hidden, one decoding step. The distilled config uses 8 denoising steps.
File sizes (HF API, decimal GB): DiT/distilled_int8 9.06, VAE/PixelUMM_FT_FP8 9.13, VAE/DCAE-V 4.28, DiT/distilled FP32 34.21, DiT/base_ar FP32 34.21, PixelUMM_FT 31.55, cinematic LoRA 6.12, robotics LoRA 12.24, PE/ 55.58 (18 shards). Whole repo 196.4 GB (182.9 GiB). Qwen2.5-VL-7B-Instruct: 16.6 GB, separate repo.
The preset reads INT8 DiT + FP8 PixelUMM + Qwen2.5-VL-7B, about 34.8 GB decimal (about 32.4 GiB). DCAE-V is not read when PixelUMM is the decoder (pipeline/base.py); counting it anyway gives about 39.1 GB. Our backend's log shows 33 GB downloaded in about 20 s (06:48:23 to 06:48:43 UTC).
tencent/Hy-Nomas-Diffusers: BF16 (unquantized, not the FP32 originals), 45.2 GB, HyNomasModularPipeline, needs github.com/jiaxiangc/diffusers@hy-nomas. Its example uses 4 steps at 704x1280 and no PE. The card says the INT8/FP8 RTX 4090 config is a separate native deployment. Not used here.
Code pins: Python >= 3.12, torch==2.10.0+cu130, transformers==4.57.1, diffusers==0.36.0, nvidia-cutlass-dsl[cu13]==4.8.0, flash-attn and flash-attn-3==3.0.0.

The single-GPU preset

configs/experiment/single_gpu_low_mem_inference.yaml sets seg_ar_attention_backend: flex ("for SM89 or below"), vae.cutlass_fp8: true, vae.cpu_offload: true, text_encoder_cpu_offload: true, dit_cpu_offload: true, dit_weight_quantization: int8, kv_cache_quantization: int8, sliding window past_frames: 18 (latent frames; one 120-frame segment is 30 latent frames, so 18 is about 3 s of video), sparse attention triton_bf16, torch_compile: true, sp_size: 1. Weights and KV are stored in INT8 and computed in BF16. The tech report (6.2) adds sparse attention: dense for the first two steps, then 25% of candidate video blocks. The standard config uses sp_size=8 and distributed PixelUMM decoding, so its default launch needs eight GPUs in one group.

The CUTLASS FP8 decoder kernel is compiled with -gencode=arch=compute_89,code=sm_89 and raises "CUTLASS FP8 backend requires SM89" on any other capability. It needs CUTLASS_PATH pointing to a CUTLASS checkout with the Ada GemmBlockwise example 94 (hymm/models/vae/pixelumm_decoder/cutlass_fp8.py). As written, the preset needs an Ada GPU. Our logs put the slower first call in a fresh process down to kernel compile and caches; one such call (84.9 s) was not slower than the warm turns.

Setup

One RTX 4090 24 GB (23.52 GiB reported by PyTorch) on our cloud (InferenceKey Cloud), driver 595.91.07 (CUDA 13.2), AMD EPYC 7513, 503 GB host RAM, 100 GB disk. Machine requested with at least 64 GB RAM.
A local InferenceKey manager bootstrapped the worker. Production was not involved.
Custom Python backend hy-nomas-world 0.1.2: apps/seg_ar_web_server.GenerationService(rank=0, world_size=1, rollout_type="seg_ar") with +experiment=single_gpu_low_mem_inference, preserve_kv_cache=True between requests, one resident world.
setup() fetches the source tarball at the pinned commit and CUTLASS v4.8.1 headers, runs snapshot_download for the preset's files and Qwen2.5-VL-7B only, puts the venv bin/ and /usr/local/cuda/bin on PATH, sets CUTLASS_PATH.
Steering prompts: one seed per world, 42 for the fox world and 7 for the keeper world; the snow retry reproduced segment 1 byte for byte. No other seeds tried.

Step by step with the SDK example

Clone inferencekey-sdk and open examples/hy-nomas-world-model-custom-backend-private-nvidia/ (commit 403c619b). Files: backend.py, publish.py, main.py, requirements.txt, README.md, .env.example.
cp .env.example .env and set INFERENCEKEY_SDK_TOKEN (ik_sdk_), INFERENCEKEY_API_KEY (ik_live_), INFERENCEKEY_PROJECT and IK_WORKER_ID. Optional: INFERENCEKEY_TENANT_ID and INFERENCEKEY_USER_TOKEN so publish.py uploads the backend and main.py turns on the chat route (otherwise do both in the dashboard).
pip install requests plus the local SDK build.
python publish.py once: packages backend.py and uploads it.
python main.py: ensure() a fixed workload (ready_timeout_secs 3600, request_timeout_secs 600), turn on the chat route, wait_until_ready(), three turns of the fox world, outputs/segment-1.mp4 to segment-3.mp4, then world.mp4 joined with ffmpeg on your side. IK_KEEP_ALIVE=1 keeps the workload.

Worker requirements: Ada (SM89) with 24 GB, driver for CUDA 13.0+ (580+), nvcc at /usr/local/cuda/bin (the FP8 kernel is built on first use), 64 GB+ host RAM, about 60 GB of disk.

A turn goes as JSON in the last user message of /v1/chat/completions:

{"step": "start", "style": "Cinematic fantasy", "perspective": "Third-person view",
 "subjects": [{"subject_id": "fox", "display_name": "a small orange fox",
               "subject_attribute": "Orange fur, white chest, and a fluffy tail."}],
 "scene": "A small orange fox stands on a stone path in a luminous forest.",
 "action": "The fox walks slowly along the path.", "seed": 42}

{"step": "continue", "session_id": "w50f42932fa5e",
 "change": "Glowing blue flowers open beside the path as the fox continues forward.",
 "action": "The fox passes the flowers and looks around."}

The answer carries session_id, segment, video_b64 (this segment's MP4, 1.1 to 2.2 MB in our runs), width, height, frames and timings (gpu_s, peak_vram_mib). Knobs in the workload env: NOMAS_VRAM_LIMIT_GIB, NOMAS_HEIGHT, NOMAS_WIDTH, NOMAS_EXTRA_OVERRIDES (Hydra overrides, for example inference.infer_steps_video=6).

Timings

Measurement	Value
`setup()` with weights on disk	18.0 to 19.0 s (pipeline load 12.1 to 13.1 s)
`ensure()` to ready, venv and weights cached	31.1 s (run 2), 19.2 s (run 3), 19.5 s (redeploy)
uv venv, cold	42 s
Weights download	33 GB in about 20 s
First segment in a fresh process	84.9 to 138.1 s
Every next segment	88.2 to 96.1 s (88.2 to 93.7 s through the platform; 96.1 s in the harness)
Denoise / KV cache write	30.8 to 38.7 s / 7.9 to 10.1 s
PixelUMM decode + MP4 encode	about 30 to 45 s (MP4 encode 2.5 to 3.4 s)

No fully cold end-to-end ready time through the worker (fresh machine, no caches) was measured.

Per segment, main run through the platform

1280x704, 120 frames, 24 fps, 8 steps, seed 42. Segment MP4s are 5.000 s, video only; world.mp4 is 360 frames, 15.000 s.

Turn	Instruction	GPU s	Round trip s	PyTorch peak MiB	Followed?
1 start	small orange fox on a stone path in a luminous forest, blue fireflies	88.36	89.3	17,774	fox, path, forest yes; glow and fireflies no
2 continue	glowing blue flowers open beside the path; fox looks around	93.05	93.6	18,056	flowers yes, not glowing
3 continue	snow begins to fall, floor turns white; fox stops and looks up	88.21	88.5	18,336	no: sunny, fox keeps walking

Other runs: harness 84.9 / 96.1 s; platform run 2 138.1 s (first call in a fresh process) / 93.7 s; keeper world 87.4 / 90.5 s (storm not followed); snow retry 85.6 / 91.3 s. Card peak 18,952 MiB (nvidia-smi, harness, two turns).

Steering count: four scene-change instructions (blue flowers, snow, snow retry with stronger wording, storm), one followed. On top of those, two actions ("stop and look up", "raise the lantern"), neither followed.

GiB are binary, slightly larger than the GB on the box, so a 24 GB card shows 23.5 GiB: 18,336 MiB is 17.9 GiB; 18,952 MiB is 18.5 GiB.

Determinism by md5: segment 1 (418955576ca2...) identical across harness, 20 GiB rung, snow retry, platform runs 2 and 3; segment 2 (0f87561f...) identical across harness, 20 GiB rung, platform runs 2 and 3.

System RAM

CPU KV archive from the backend logs: cpu_archive_gib=5.881 after 1 segment, 11.763 after 2, 17.644 after 3 (total_frames 30/60/90, 18 latent frames kept on the GPU). Linear, about 5.88 GiB per segment. No world went past 3 segments. Process RSS was not measured. 64 GB in the example is a provisioning choice, not a measured minimum.

VRAM ladder

Caps emulated on the same 4090 with torch.cuda.set_per_process_memory_fraction (NOMAS_VRAM_LIMIT_GIB).

GPU or cap	Config	Peak VRAM	Time per segment	Output OK	Real or emulated
RTX 4090 24 GB	preset, 1280x704, 8 steps, through the platform	18,336 MiB PyTorch; card 18,952 MiB	88 to 94 s (96.1 s in the harness)	yes	real
20 GiB cap	same, start + continue	18,056 MiB	137.6 s first / 90.5 s	yes, byte-identical MP4s	emulated
16 GiB cap	same			no: `torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB ... 15.98 GiB is allocated by PyTorch ... 16.00 GiB allowed` in `_encode_prompt` (bf16 Qwen2.5-VL-7B moved to the GPU)	emulated

Not tried: an 18 GiB cap, a real 20 GB Ada card (RTX 4000 Ada), an fp8 or CPU text encoder, lower resolution, fewer steps, non-Ada cards (RTX 5090 sm_120, A40 sm_86, H100), and other Ada cards. The 20 GiB cap covered two segments. The tech report's 20.53 GiB peak is for longer segments (100 s in seven segments); the report ties peak memory to segment length, as the GPU history is bounded by the sliding window. Many 5 s turns on a real 20 GB card are untested.

Problems and status

OOM with the online worker's decode flags. inference.seg_ar_segment_vae_decode=true + model.vae.stable_prefix_streaming=true (set by run_seg_ar_gpu_worker.sh) decode with the DiT resident: torch.OutOfMemoryError ... Tried to allocate 826.00 MiB ... 22.50 GiB is allocated by PyTorch. Dropped both flags; decode runs after denoising. Worked around in the example, not filed upstream.
torchvision mismatch. timm pulled PyPI torchvision 0.25.0 (CUDA 12) next to torch cu130: operator torchvision::nms does not exist. Fixed by pinning the cu130 wheel.
Ninja is required to load C++ extensions during the FP8 kernel JIT: the venv bin/ is not on PATH. Fixed in the backend.
8 MB message limit. An answer carrying the segment plus the whole-world MP4 finished on the worker but never reached the client (NATS max_payload 8,388,608). The world video was removed from the answer; the client joins segments. Longer outputs need object storage, not inline base64. Open in the platform.
No text2video custom backend type. The packager rejects task_type 'text2video', and the workload modality must match the backend. The example uses text2image for both and returns MP4 on the chat route. Open in the platform.
Seed per world. A continue turn without a seed got 400 {"error": "interactive continuation changed seed: existing=7, requested=42"}. Fixed in backend 0.1.2, which keeps the start seed.
300 s synchronous limit (manager DEFAULT_TIMEOUT_MS) forces one segment per call: a 90 s segment fits, a multi-segment render in one call would not. Async jobs deliver by webhook only. Open in the platform.
Weak steering in our runs: scene-wide changes and actions not followed, start prompts partly missing. Untested whether Nomas-PE fixes it.
Camera paths not tested ("no camera_trajectory was provided"), and no world went past 3 segments.

Demo video

Correction for the video: its memory insert says "18.3 GiB" peak. It reads 18,336 MiB as thousands; the correct figures are 17.9 GiB PyTorch peak and 18.5 GiB card peak.

176 s, 1920x1080, English subtitles. It includes the authors' promo clips, credited on screen, and their benchmark figures marked as not reproduced by us, then our fox and keeper worlds through the endpoint.

Authors' benchmark (not reproduced)

Hy-Nomas-Bench: 280 cases in four benchmarks, pairwise human votes. Bench-1 multi-turn text: prompt-following win rate 65.0% against Happy Oyster (Directing) and 75.0% against LingBot-World v2; physical plausibility 45.0% and 68.3%. Spatial memory comparable to Happy Oyster, trails Genie 3.

Sources

Model card: https://huggingface.co/tencent/Hy-Nomas
Code: https://github.com/Tencent-Hunyuan/Hy-Nomas
Paper: https://arxiv.org/abs/2610.36729
Diffusers version: https://huggingface.co/tencent/Hy-Nomas-Diffusers
Text encoder: https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct
SDK example: https://github.com/inferencekey/inferencekey-sdk/tree/main/examples/hy-nomas-world-model-custom-backend-private-nvidia
Where we spotted it: https://www.youtube.com/watch?v=EfyQOAJjx1A