Benchmarking small CPU text-to-speech models for local deployment
A November 2026 benchmark compares latency, memory, and quality of lightweight TTS models on a single-threaded AMD EPYC CPU.
Engineers building offline or edge AI features now have concrete data on how small text-to-speech models perform on standard server hardware. A detailed benchmark published in November 2026 evaluates twenty models on a single-threaded AMD EPYC Rome CPU, measuring latency, memory usage, and audio quality without GPU acceleration.
What happened
The test evaluated a wide range of open-weight TTS systems, from tiny experimental models to larger production-ready engines. The goal was to identify which models offer the best balance of speed and naturalness for CPU-only environments. The benchmark ran each model in isolation using a consistent Python harness on Ubuntu 24.04, ensuring that results reflected real-world single-core performance rather than optimized multi-threaded bursts.
Models were tested on five specific text samples, including short greetings, long paragraphs, and complex number sequences. The evaluation focused on median latency for the first audio chunk, overall processing speed, peak memory consumption, and intelligibility. Intelligibility was measured using Word Error Rate (WER) via Whisper large-v3, while naturalness was scored using the UTMOS22 predictor. The results highlight significant differences in efficiency, with some models generating audio faster than real-time while others struggled to keep up even on powerful server-grade CPUs.
How it works
The benchmark infrastructure used a KVM virtual machine with 16 vCPUs but restricted most tests to a single thread to simulate constrained edge devices or background tasks. Each model ran in its own process with local files, avoiding network latency. The test measured "first audio" time, which is the delay between the API call and the start of playback, a critical metric for interactive applications.
Performance was quantified using Real-Time Factor (RTF), where a value below 1.0 means the model generates audio faster than it takes to speak it. Memory usage was tracked as peak process memory, and quality was assessed objectively. WER measures how accurately a speech-to-text system can transcribe the output, indicating clarity, while UTMOS predicts human-like naturalness on a scale of 1 to 5. This dual-metric approach helps developers distinguish between models that are merely fast and those that produce usable, pleasant speech.
Key details
- Fastest latency: Linnet en-35m achieved the lowest first-audio time at 70 ms, followed closely by Phonon 31m at 82 ms.
- Highest efficiency: SupraTTS-1.0-Preview reached a speed of 74× real-time on one thread, though it lacks streaming support.
- Lowest memory: Mach Boom used only 107 MB of peak memory, making it suitable for highly constrained environments.
- Best naturalness: nano-edge-tts (Kokoro) scored highest on UTMOS at 4.50, despite a slower 550 ms first-audio time.
- Multilingual support: Paradee-8M v1.3 supports English, Hindi, French, Spanish, and German, with a moderate 261 ms latency.
- License variety: Most top performers use Apache-2.0 or MIT licenses, though some like Underdog Woof are restricted to non-commercial use.
Why it matters
For developers deploying AI on edge devices, laptops, or cost-sensitive cloud instances, GPU access is not always guaranteed. These results show that high-quality speech synthesis is viable on CPU alone, provided the right model is chosen. Models like Linnet and Phonon offer near-instant response times, which is essential for voice assistants and interactive tools where lag breaks the user experience.
Memory constraints are often a harder bottleneck than compute power in embedded systems. The wide range of memory footprints, from 107 MB to over 2 GB, allows engineers to match models to specific hardware limits. Additionally, the discrepancy between claimed performance in model cards and measured results underscores the need for independent benchmarking. Claims often rely on optimal hardware or partial metrics, whereas this test provides end-to-end data relevant to production deployment.
What you can do
- Prioritize Linnet en-35m or Phonon 31m if your application requires sub-100 ms latency for interactive voice features.
- Choose Mach Boom or SmollerTTS if your deployment environment has strict memory limits under 200 MB.
- Use nano-edge-tts (Kokoro) when audio naturalness is the primary concern and slightly higher latency is acceptable.
- Verify license compatibility before integration, especially for commercial products, noting that some high-performing models are non-commercial.
- Test multilingual requirements early, as many fast models like Linnet are English-only, while others like Paradee offer broader language support.
- Benchmark your specific target hardware, as performance on an AMD EPYC server may differ significantly from ARM-based edge devices.



