A private ChatGPT for fifty teams, and one bill you can split
How one private AI assistant serves fifty teams through one address, and how its fixed hourly cost becomes a bill per team. A demo, with its limits.
InferenceKey
Posts
-
-
Pocket TTS in Castilian Spanish: a voice that runs on CPU
We fine-tuned Kyutai's Pocket TTS for Castilian Spanish. It runs on a plain CPU, with samples in Spanish, English, French and German.
-
EmbeddingGemma 2: one small open model to search text, photos, video and speech
Google's EmbeddingGemma 2 puts text, images, video and audio in one search space. It loaded in 1.4 GB on our 16 GB RTX 2000 Ada.
-
Kandinsky 6.0: open video with sound, priced in minutes per clip
Kandinsky 6.0 makes 5-second clips with synced audio under MIT. Minutes per clip on seven GPUs, and why every published run uses offload.
-
Iris-3B: an open image model that draws its own pixels, about two seconds a picture
Iris-3B is an open image model that draws pixels itself. We ran its fast Turbo version on InferenceKey: about 2 seconds a picture on an A40.
-
A model that would need 1.5 TB of VRAM in FP16, running in just 192 GB
DeepSeek-V4.1-Flash, a 763B model, would take 1.5 TB in FP16. We run it on two 96 GB cards for 19 developers, thanks to open source.
-
AREX-2: a 27B agent model that serves like Qwen3.8-27B
BAAI's Apache-2.0 AREX-2 shares Qwen3.8-27B's architecture. What our base-model sglang runs say about its VRAM, quants and GPUs, and what is untested.
-
An 80B coding model on one rented A100, driven by Claude Code
Qwen3 Coder Next (80B, AWQ 4-bit) on vLLM, one A100 80 GB at $1.79/h, an API key for one developer and Claude Code pointed at it. Commands, numbers, limits.
