Apple SpeechAnalyzer Benchmark: Performance vs Whisper and SFSpeechRecognizer

Apple's new SpeechAnalyzer API is the most accurate on-device speech engine for English transcription among those tested, surpassing both the legacy SFSpeechRecognizer and several Whisper models. In benchmarks conducted on an Apple M2 Pro, SpeechAnalyzer achieved a Word Error Rate (WER) of 2.12% on clean speech and 4.56% on noisy speech, running approximately three times faster than Whisper Small.

Accuracy Comparison: SpeechAnalyzer vs. Competitors

SpeechAnalyzer provides a significant accuracy leap over Apple's previous on-device solution and outperforms common Whisper configurations for English audio.

Engine	test-clean WER	test-other WER	Model size
Apple SpeechAnalyzer (iOS/macOS 26)	2.12%	4.56%	system
Whisper Small (WhisperKit CoreML)	3.74%	7.95%	~460MB
Whisper Base	5.42%	12.51%	~140MB
Whisper Tiny	7.88%	17.04%	~40MB
Apple SFSpeechRecognizer (legacy)	9.02%	16.25%	system

Migration from SFSpeechRecognizer

Developers should migrate from the legacy SFSpeechRecognizer to SpeechAnalyzer immediately. The new API reduces the word error rate by 3.5x to 4x on the same audio. Beyond raw accuracy, SpeechAnalyzer produces punctuated and cased text, whereas the legacy engine's output is rougher.

SpeechAnalyzer vs. Whisper

For English transcription on Apple hardware, SpeechAnalyzer is now the strongest on-device option. It beats Whisper Small—the largest model tested—by a comfortable margin while requiring roughly one-third of the compute time per second of audio. However, Whisper retains two primary advantages: it supports significantly more languages (SpeechTranscriber supports approximately 30 locales) and it is cross-platform.

Performance and Speed

All tested engines operated faster than real-time on the M2 Pro (macOS 26.5.1), with transcription speeds ranging from 12x to 40x. SpeechAnalyzer specifically demonstrated a 3x speed advantage over Whisper Small per second of audio while maintaining higher accuracy.

Methodology and Validation

To ensure the results are grounded and reproducible, the benchmark employed the following standards:

• LibriSpeech Corpus: The test used the standard LibriSpeech dataset (test-clean and test-other splits) to allow for direct comparison with OpenAI's published Whisper numbers.
• Validation against OpenAI: The benchmark's Whisper results closely matched OpenAI's published WER, with only a small, consistent positive offset attributed to CoreML quantization and a stricter text normalizer.
• On-Device Verification: All engines were forced to run on-device to prevent cloud-based results from skewing the data. SFSpeechRecognizer was explicitly configured to refuse cloud fallback.
• Text Normalization: Both the reference text and engine outputs passed through the same normalizer (handling casing, punctuation, and digits-to-words) to ensure engines were not penalized for formatting.

Developer Insights and Limitations

Technical Limitations

• Language Scope: The benchmark is limited to English read speech. It does not account for the 100+ languages supported by Whisper.
• Audio Type: The tests used audiobook speech rather than spontaneous meeting audio, which includes accents and multi-speaker overlap.
• Hardware: Speed results are specific to the M2 Pro chip; while accuracy should transfer across Apple Silicon, timing will vary by device.

Practical Implementation

Developers using the API have noted that SpeechAnalyzer supports streaming, allowing users to see transcriptions in real-time—a significant UX improvement over models that require full audio blobs for processing. Additionally, using the system API reduces app bundle size because the model is managed by the OS rather than bundled within the application.

One noted drawback is that SpeechAnalyzer requires a separate model download per language, which can complicate the transcription of multi-lingual audio if the languages are not known in advance.

Community Perspectives

While the benchmark highlights the strength of SpeechAnalyzer for English, community members have raised points regarding other state-of-the-art models:

"Whisper is the wrong model to benchmark against, or rather, there are better models that are state of the art now like Nemotron and Parakeet both by Nvidia, as well as Mistral's Voxtral and Cohere Transcribe."

Other users have noted that while SpeechAnalyzer is highly efficient, they still prefer Whisper Large-V2 or Large-V3 Turbo for specific use cases, such as math lectures, where they prioritize absolute accuracy over real-time generation speed.