Nvidia's Nemotron 3.5 Transcribe Runs a 30B Voice Model on One GPU

Nvidia has released Nemotron 3.5 Transcribe, an open-weight speech model that transcribes, translates, and identifies speakers across up to 40 languages while running on a single GPU, according to The Decoder. The model uses a 30-billion-parameter language-model backbone but activates approximately 4.3 billion per processing step, a design choice that lets it run on hardware including a single RTX PRO 6000 Blackwell GPU in Nvidia’s official NVFP4 demonstration. Nvidia paired the release with Granary v2, an 844,000-hour speech dataset, and fine-tuning tools aimed at specialized fields like medicine.

What Happened

Nemotron 3.5 Transcribe is a speech-augmented language model that takes text instructions alongside audio, letting the same recording become a verbatim transcript, a speaker-labeled log, an English translation, or a summary depending on the prompt, according to The Decoder. Nvidia says the model transcribes 40 languages, translates 31 of them to or from English, identifies up to eight speakers in 13 languages, and handles recordings as long as an hour.

Nvidia’s official research page describes a speech encoder, a frozen diarization model and a Nemotron 3.5 Lightning 30B-A3B mixture-of-experts language model, with approximately 4.3 billion active parameters. The company offers BF16, FP8 and NVFP4 checkpoints; the NVFP4 version is intended for Blackwell GPUs. Total model size and active parameter count describe different things, so a 30-billion-parameter headline should not be read as 30 billion parameters being activated on every processing step.

Nvidia reports a mean word error rate of 4.71 percent across nine public English test sets and 4.62 percent on the multilingual Open ASR Leaderboard. These are the company’s reported evaluation results, not a guarantee for a particular customer recording. The Decoder reports a separate 2.81 percent result in Artificial Analysis’ comparison. Different test sets and evaluation conventions produce different scores; these figures should not be combined into one universal accuracy claim or compared as though they measured identical audio.

The speaker-recognition results stand out more sharply. According to The Decoder, VoiceArena tested 14 systems on overlapping-speaker conversations and ranked Transcribe first, missing only 8.32 percent of words compared with a roughly 31 percent median miss rate among competitors. On David AI's 21-language speaker attribution benchmark, it posted the lowest error rate, edging out Nvidia's own dedicated diarization model.

The practical limitation is that this release concerns recorded audio rather than a live conversational stream, according to TechCrunch. That distinction matters when selecting a model for meeting archives, interview transcription or document preparation. A recorded-audio tool can be evaluated against completed sessions, while a live agent needs an additional assessment of response delay and interaction behavior. This article does not claim that Nemotron 3.5 Transcribe replaces a real-time voice system simply because both use speech models.

Nvidia also demonstrated domain adaptation using the Granary v2 dataset. According to The Decoder, a more precise prompt reduced the error rate on Indian doctors’ and medical students’ recordings from 30.8 to 23.8 percent, and a subsequent fine-tuning step reduced it to 11.8 percent, a process Nvidia says took about 14 GPU hours on a single GB300. The Decoder adds that this specialization carries tradeoffs: general English accuracy dipped slightly, and a dedicated medical model still beat it on brand-name drug recognition.

Why It Matters

MachineHours analysis: Total parameter count alone is a poor guide to the cost of a transcription service. Nvidia describes approximately 4.3 billion active parameters, while its language-model backbone has a much larger total parameter count. The distinction helps explain the efficiency pitch, but it does not eliminate the memory needed to store model weights. Precision format, serving software and the length of recordings remain part of a deployment decision. Nvidia’s official demonstration uses an RTX PRO 6000 Blackwell GPU with an NVFP4 checkpoint and DFlash2; it should not be treated as a performance promise for every single-GPU configuration.

Background

The efficiency play here connects to a longer thread in chip design: the move toward lower-precision number formats like NVFP4 that squeeze more usable compute out of the same silicon. That shift depends on advanced manufacturing processes concentrated at a handful of foundries, a dependency explored in TSMC's Monopoly. The broader push to run capable AI models on single consumer or workstation GPUs, rather than large clusters, traces back to the architectural changes described in The GPU Revolution, which reshaped chips originally built for graphics into general-purpose AI accelerators.

What to Watch

Nvidia’s official page reports up to 2,238 seconds of audio processed per second on a single RTX PRO 6000 Blackwell GPU using its Nemotron Transcribe NIM, NVFP4 checkpoint and DFlash2. The configuration matters as much as the headline throughput. MachineHours analysis: Developers should evaluate the actual checkpoint and serving stack they plan to use, with representative accents, terminology and speaker overlap, rather than extrapolating from a vendor’s maximum result. The release includes fine-tuning tools, but adaptation should be checked for regressions on general audio as well as improvements in the target domain.