Offline live subtitles with Whisper and M2M100: how it works and what it costs
Offline live subtitles with Whisper and M2M100: how it works and what it costs Cloud captioning is convenient until you look at the bill, the latency, or the privacy policy. For streamers translating Russian speech into English in real time, a local pipeline can be faster, cheaper, and fully priva

te. Here is how ours works. Audio β VAD (voice activity detection) β Whisper (speech recognition) β local translation β WebSocket β OBS overlay. Capture β microphone or system audio (WASAPI loopback on Windows).
VAD β detects phrase boundaries so we don't feed silence or game noise into the model. Whisper β transcribes speech; for RUβEN it can translate directly with task=translate. Translation model β for other language pairs we use local models: M2M100 (MIT) or the fast Helsinki opus-mt-ru-en (CC-BY-4.0). Output β a WebSocket stream rendered as a browser source in OBS, with configurable font, colors and line history.
Filters β hallucination and repetition filters keep captions readable. On a modern NVIDIA GPU we see roughly 0.3β0.7 s per phrase, including translation. On CPU it still works, just slower (1β2 s and more depending on the model). For live captions, predictable latency matters more than "maximum accuracy": medium on a GPU is the sweet spot for most streams.
NLLB is a great model, but its license (CC BY-NC 4.0) forbids commercial use. Shipping it inside a paid product is not an option, so in Stream Subs 1.8 we removed NLLB and kept: M2M100-418M (MIT) β local translation for 15 languages, ~0.5 s per phrase; Helsinki opus-mt-ru-en (CC-BY-4.0) β very fast RUβEN (120β230 ms); Whisper translate β one model, simplest path; optional LLM via API β better quality when the user explicitly opts in (text leaves the machine, so it is off by default). Cloud captions Local (Whisper + M2M100) Latency depends on your uplink 0.3β0.7 s on GPU Cost subscription, minute limits one-time purchase, no limits Privacy audio goes to someone's server everything stays on your machine Works offline no yes Hardware none NVIDIA GPU recommended For a streamer doing 10+ hours a month, a cloud subscription usually costs more in the first year than a one-time license β and the audio never leaves their PC. Stream Subs 1.8 adds a CPU build for machines without a discrete GPU, session statistics (lines, translations, average latency), presets for stream/webinar/recording scenarios, and a choice of translation engine.
It is open source (MIT): github.com/TenLux-dev. Product page: tenlux.tech/stream-subs VAD quality matters more than model size. Bad segmentation ruins captions faster than a small model. Filters are mandatory.
Whisper will happily invent phrases on silence. Licenses decide architecture. One model with a non-commercial license can force a rewrite. CPU mode is not a gimmick.
Many streamers have no discrete GPU; supporting them doubled the audience. If you are building something similar, or you stream and want to try it, feedback is welcome β we are a small studio, and real use cases shape the roadmap more than feature lists.
Key Takeaways
- β’Here is how ours works.
- β’This story was reported by Dev.to, covering developments in the dev space.
- β’AI advancements continue to reshape industries β read the full article on Dev.to for complete coverage.
π Continue reading the full article:
Read Full Article on Dev.to βShare this article



