NVIDIA releases open-weight Nemotron 3.5 Transcribe for promptable speech processing

NVIDIA has released Nemotron 3.5 Transcribe, an open-weight speech model that combines transcription, speaker attribution, and translation into a single checkpoint. Available now on Hugging Face and ModelScope, the model supports 40 languages for transcription and allows developers to guide output using natural language prompts. This release marks a significant step toward customizable, high-accuracy speech processing that can run locally or scale in data centers.

What happened

The new model handles recordings up to one hour long and can attribute speakers in 13 languages, labeling up to eight distinct voices. It translates 31 languages to or from English and supports audio question answering, though the latter is trained primarily in English. Developers can use natural language instructions to specify output formats, such as requesting speaker labels, translations, or specific formatting for domain jargon like medical terms.

Nemotron 3.5 Transcribe leads the Hugging Face Open ASR Leaderboard for English among open-weight models with a 5.33% average word error rate (WER). It also ranks first on David AI’s 21-language diarization leaderboard and tops VoiceArena’s 14-system comparison for speaker attribution. These benchmarks highlight its ability to handle overlapping dialogue and fast-paced speech, as demonstrated in tests using scenes from the 1940 film His Girl Friday.

Alongside the model, NVIDIA released Granary v2, a multilingual data curation pipeline used to train Transcribe. Granary v2 provides reusable tools for preparing speech data, including generating aligned transcript variants and quality metadata. The public manifest covers 463,062 hours of audio across 48 languages, offering developers a foundation for building their own specialized speech datasets.

How it works

Transcribe uses a hybrid architecture that pairs a speech encoder with a Mamba–Transformer mixture-of-experts language model. This design ensures that only about 4 billion of its 30 billion parameters are active per token, placing it on the accuracy–active-parameter frontier. Speaker cues are integrated directly into the generation process via Sortformer, allowing the model to tag speakers during transcription rather than relying on a separate post-processing step.

For domain adaptation, NVIDIA provides NeMo Speech fine-tuning tools. A medical case study showed that fine-tuning reduced word error rate by 45.8% on held-out speakers compared to prompting alone. The process involves generating synthetic speech from text targets, such as converting "HbA1c" into spoken forms, and training the model with LoRA adapters. This allows the system to learn specific vocabulary and formatting conventions without retraining the entire model.

Granary v2 supports this workflow by producing three transcript forms for each recording: text-normalized verbatim, inverse text normalization with disfluencies, and clean inverse text normalization. Developers can use these variants to define exactly how they want the model to handle numbers, units, and filler words. The pipeline filters out low-quality audio, ensuring that training data meets high fidelity standards.

Key details

• Supports transcription in 40 languages and translation for 31 languages to or from English.
• Achieves 5.33% average WER on English, leading open-weight models on the Hugging Face leaderboard.
• Attributes up to eight speakers in recordings up to one hour long with low diarization error rates.
• Uses approximately 4 billion active parameters per token via a mixture-of-experts architecture.
• Granary v2 dataset manifest includes 463,062 hours of audio across 48 languages.
• Quantized versions for local deployment on RTX and Jetson hardware are coming soon.

Why it matters

For engineers building voice-enabled applications, the ability to prompt the model for specific output formats reduces the need for complex post-processing scripts. Instead of writing custom regex rules to clean up transcripts or separate speakers, developers can instruct the model to produce the exact format they need. This flexibility is crucial for handling domain-specific jargon in fields like healthcare or law, where standard transcription often fails.

The open-weight nature of the model also addresses data privacy concerns. Organizations can run Transcribe locally on their own infrastructure, keeping sensitive audio data within their secure environments. With quantized versions arriving for consumer and edge hardware, high-quality speech processing becomes accessible without relying on third-party APIs. This shift empowers teams to build more robust and private voice solutions.

What you can do

• Download the Nemotron 3.5 Transcribe checkpoint from Hugging Face or ModelScope to test locally.
• Use the provided natural language prompts to experiment with different output formats and speaker labeling.
• Explore the Granary v2 manifests to understand the structure of high-quality speech training data.
• Follow the NeMo Speech fine-tuning guide to adapt the model to your specific domain vocabulary.
• Deploy the model using vLLM or SGLang-Omni for high-throughput inference in production environments.
• Evaluate the model on your own audio samples to identify areas where domain adaptation may be needed.