Cohere Transcribe benchmarks show accuracy gaps in meeting audio

Independent engineers at InferenceBench tested Cohere’s cohere-transcribe-03-2026 model on H100 hardware in October 2026. The results confirm high accuracy for clean studio speech but reveal significant performance drops on real-world meeting audio compared to the vendor's published metrics.

What happened

Cohere released cohere-transcribe-03-2026 in March 2026, marketing it as a high-performance speech-to-text solution. The company claimed an average Word Error Rate (WER) of 5.42% across four standard datasets, positioning it as a competitor to OpenAI Whisper Large v3 and hosted services like Deepgram. To verify these claims, the InferenceBench team ran the model on NVIDIA H100 GPUs using configurations of one, two, four, and eight cards.

The independent evaluation used the same four datasets cited by Cohere: LibriSpeech-clean, LibriSpeech-other, AMI-IHM, AMI-SDM, and VoxPopuli. While the model achieved a strong 1.46% WER on the clean LibriSpeech subset, the overall average across all four datasets was 13.65%. This figure is roughly 2.5 times higher than the 5.42% advertised by Cohere. The discrepancy stems primarily from poor performance on the AMI meeting datasets, where error rates exceeded 25%.

Additionally, the multilingual evaluation component of the test failed entirely due to a benchmarking infrastructure issue. The test harness could not load audio samples for any of the 14 attempted languages, returning zero working samples. While this appears to be a testing bug rather than a model failure, it leaves Cohere’s multilingual capabilities unverified by this specific independent review.

How it works

Cohere Transcribe 03-2026 is an encoder-decoder transformer architecture trained on a large multilingual corpus. It is designed to run efficiently on single-GPU setups for production workloads, though it can scale across multiple GPUs for increased throughput. The model is available via API and as a downloadable checkpoint under a commercial license.

In terms of speed, the model achieves a Real-Time Factor (RTFx) of approximately 12x across all tested GPU configurations. This means it can transcribe 30 seconds of audio in about 2.5 seconds of compute time. Notably, adding more GPUs increases the total number of concurrent jobs the system can handle but does not reduce the latency for individual requests. The per-request speed remains constant regardless of whether one or eight H100s are used.

The model also demonstrates robust behavior when processing non-speech audio. Unlike some earlier automatic speech recognition systems that hallucinate text during silence or noise, Cohere Transcribe returns empty or minimal outputs when encountering music, white noise, or silent segments. This stability extends to degraded signal conditions, where performance degrades gracefully rather than collapsing into nonsensical transcriptions.

Key details

• Clean speech accuracy: The model achieved a 1.46% WER on LibriSpeech-clean, matching top-tier public leaderboard results.
• Meeting audio struggle: WER spiked to 17.74% on AMI-IHM (headset mic) and 29.17% on AMI-SDM (distant mic), dragging the overall average up.
• Cost efficiency: Self-hosting on a single H100 costs approximately $0.21 per hour of audio, which is cheaper than most hosted APIs at scale.
• Multilingual gap: The independent test could not validate multilingual performance because the test harness failed to load the required dataset samples.
• Throughput scaling: A single H100 can process around 600 hours of audio per day, with multi-GPU setups adding capacity but not speed per clip.
• Robustness: The model avoids hallucinating text during silence or noise, providing empty outputs instead of confabulated speech.

Why it matters

For engineering teams building voice-enabled products, the gap between marketing claims and real-world performance is critical. If your application involves podcast transcription, dictation, or studio-quality recordings, Cohere Transcribe performs exceptionally well. However, for use cases involving conference calls, meeting recordings, or far-field microphones, the error rate may be prohibitively high without additional preprocessing. Developers must account for this domain specificity when choosing between self-hosted models and managed APIs.

The cost analysis also shifts the decision-making framework for high-volume users. At roughly $0.21 per hour of audio on self-hosted infrastructure, the model becomes economically viable for companies processing over 2,000 hours of audio monthly. Below that threshold, the operational complexity of managing GPU infrastructure may outweigh the savings compared to hosted solutions like Deepgram or AssemblyAI. Understanding the break-even point helps technical founders allocate budget effectively between compute resources and engineering time.

What you can do

• Evaluate your primary audio source; if it involves overlapping speech or distant microphones, plan for higher error rates or implement voice activity detection chunking.
• Calculate your monthly audio volume to determine if self-hosting on H100s offers cost savings over hosted APIs like Deepgram or AssemblyAI.
• Test the model with your specific data samples before committing, especially if multilingual support is a core requirement, as independent verification is currently lacking.
• Use single-GPU deployments for initial integration since adding more GPUs will not improve per-request latency, only total throughput.
• Ensure input audio is resampled to 16 kHz mono before processing to avoid silent degradation in quality.
• Monitor for hallucinations in noisy environments, although this model shows improved robustness compared to older architectures.