Local Speech To Text on M5 Max: Whisper large-v3 vs Parakeet on MLX Benchmarked (2026)

Speech to text is the local inference workload people underestimate. It looks solved because OpenAI Whisper runs everywhere and the accuracy is good, so teams grab large-v3, wrap it in a queue, and move on. Then the product needs live captions, or it needs to run on the same machine that is already serving a language model, or it needs to transcribe forty hours of support calls overnight, and the single model choice that felt fine starts to hurt. The truth is that the two best on device transcription models in 2026 are built on opposite architectures, and the architecture is the whole decision. We loaded both on a 64GB M5 Max under MLX to show what each one actually does.

The two models are Whisper large-v3, an autoregressive encoder decoder from OpenAI, and NVIDIA Parakeet, a FastConformer transducer. Whisper reads a chunk of audio and generates a transcript token by token, the same way a language model writes text. Parakeet processes audio through a convolutional conformer encoder and emits tokens through a Token and Duration Transducer that can fire as soon as it has heard enough. One is a decoder that thinks in whole windows. The other is a streaming machine that keeps up with the microphone. That single difference propagates into speed, latency, language coverage, and how the model behaves when the audio is messy.

The Two Numbers That Matter: Accuracy And Real Time Factor

Real time factor, RTF, is the metric to internalize. An RTF of 10x means the model transcribes sixty seconds of audio in six seconds. Higher is faster. The second number is word error rate, WER, the fraction of words the model gets wrong on a reference set. Lower is better. Here is the shape of both on a 64GB M5 Max, MLX backends, single stream, clean English audio. Treat the throughput figures as our test results on this specific machine, not as guarantees for yours.

Model	Architecture	Params	Approx WER, clean English	Real time factor, M5 Max	Languages
Whisper large-v3	Autoregressive encoder decoder	about 1.5B	5 to 8 percent	12 to 18x	99
Whisper large-v3-turbo	Encoder decoder, pruned decoder	about 0.8B	6 to 9 percent	20 to 30x	99
Parakeet TDT 0.6B v3	FastConformer transducer	about 0.6B	6.3 percent avg, near 6 on clean	100x and up	25 European

Read the RTF column first. Parakeet is not a little faster than Whisper, it is an order of magnitude faster, and that gap is architectural rather than a matter of tuning. Whisper generates its transcript autoregressively, one token conditioned on the last, so decode is a sequential loop bound by the same memory bandwidth ceiling that governs local language model decode, the ceiling we mapped in the M5 Max quantization benchmarks. Parakeet's transducer does far less sequential work per second of audio, so the FastConformer encoder does the heavy lifting once and the emission is cheap. On clean English the WER is close enough between them that speed, not accuracy, is the deciding axis.

Where Whisper large-v3 Earns Its Place

Whisper wins on breadth and resilience. It covers ninety nine languages and it can translate as it transcribes, turning Spanish audio into English text in one pass, which Parakeet does not do. It was trained on a vast and deliberately noisy corpus, so it holds up on accented speech, background noise, crosstalk, and low quality phone audio far better than a model tuned for clean input. If your audio is real world messy, a voicemail, a field recording, a conference room with three people talking over each other, Whisper degrades gracefully where a cleaner model falls apart.

The autoregressive decoder is also why Whisper produces good segment timestamps and coherent punctuation. It is effectively writing well formed text, so it inserts sentence boundaries and casing the way a language model would. The cost is the one the RTF column already showed. Because decode is sequential, Whisper cannot stream in the true sense. It works on thirty second windows, so the earliest a caption can appear is after a window has been buffered and decoded, which puts a floor of a second or more on latency even when the RTF is high. For batch transcription that floor is invisible. For live captions it is the entire problem.

large-v3-turbo is the pragmatic default when you want Whisper accuracy with less of the speed penalty. It keeps the full encoder and prunes the decoder down, which trades a sliver of accuracy on hard audio for a meaningful throughput gain. On the M5 Max it lands in the twenty to thirty times real time range, which for overnight batch work over a large archive is the difference between a job that finishes before morning and one that does not.

Where Parakeet Changes What Is Possible

Parakeet is the model to reach for when latency is the product. The transducer emits tokens as the audio streams in, so you can render words on screen while the speaker is still talking, with an end to end delay measured in the low hundreds of milliseconds rather than the seconds Whisper needs to fill a window. Live captioning, voice commands, real time meeting transcription, and any interface where the user is watching text appear are all Parakeet territory. It also posts a lower word error rate than Whisper on clean English while running at a fraction of the compute, which is the combination that makes it feel almost free to run.

That efficiency compounds when the machine is doing other work. On a 64GB M5 Max you are often serving a language model in the same unified memory pool, and a 0.6B transducer running at a hundred times real time barely dents the memory bandwidth budget the language model needs, where a 1.5B autoregressive Whisper decode competes for it directly. If your product transcribes and then reasons over the transcript with a local model, Parakeet leaves far more headroom for the second stage, the same unified memory arithmetic that governs running a vision model alongside a language model.

The limits are real and you should size for them. Parakeet v3 covers twenty five European languages, not the ninety nine Whisper spans, and it does not translate. It is tuned for cleaner input, so on heavily accented or noisy audio the accuracy gap that favors it on clean speech can narrow or invert. It ships open weights under a permissive license, which for a commercial product matters as much as the benchmark, but check that the languages your users actually speak are in the covered set before you commit.

Running Both Under MLX

Both models have first class MLX ports, which is what makes this a real choice on Apple Silicon rather than a CUDA only story. mlx-whisper loads the large-v3 or turbo weights and runs the encoder and decoder on the GPU through Metal. parakeet-mlx loads the FastConformer transducer the same way. A minimal decision wrapper looks like this.

# Route audio to the right local model by requirement, not by habit.
def pick_asr(streaming: bool, languages: list[str], noisy: bool) -> str:
    european = {"en", "es", "fr", "de", "it", "pt", "nl", "pl"}
    needs_wide_lang = any(l not in european for l in languages)
    if streaming and not needs_wide_lang and not noisy:
        return "parakeet-tdt-0.6b-v3"      # low latency, clean, European
    if needs_wide_lang or noisy:
        return "whisper-large-v3"          # breadth and resilience
    return "whisper-large-v3-turbo"        # fast batch, Whisper accuracy

# Example: live English captions in a quiet room -> Parakeet
print(pick_asr(streaming=True, languages=["en"], noisy=False))
# Example: overnight batch of multilingual support calls -> Whisper
print(pick_asr(streaming=False, languages=["en", "hi", "ar"], noisy=True))

The branch that trips teams up is the first one. A streaming, clean, English use case defaults to Whisper out of habit and inherits the windowing latency for no reason, when Parakeet would give lower latency and lower WER at less compute. The reverse mistake is reaching for Parakeet on multilingual or noisy audio because the benchmarks looked good, then watching accuracy sag on exactly the input the leaderboard did not represent.

When This Applies To Your Stack

If you are putting transcription behind a product feature on Apple Silicon, choose the model by the shape of the workload, not by which name you recognize. Streaming interfaces, voice control, and live captions in clean European language audio point to Parakeet, whose transducer was built to keep up with the microphone and leaves memory bandwidth free for whatever runs next. Wide language coverage, translation, or resilient transcription of messy real world audio points to Whisper large-v3, with turbo as the fast batch variant when the language breadth matters but the streaming does not. The two are not competitors so much as tools for different jobs, and the mistake is owning one hammer.

If your team is designing an on device audio pipeline and wants the model choice, the memory budget, and the latency targets designed against real hardware rather than a spec sheet, Contra Collective builds local AI infrastructure that fits the machine it runs on. Speech to text on Apple Silicon rewards picking the architecture that matches the workload, and punishes picking the one that matches your assumptions. If you are also generating audio, the same architectural split shows up on the local text to speech side.

FAQ

Is Whisper or Parakeet more accurate for local transcription? On clean English they are close, with Parakeet posting a slightly lower word error rate near six percent. On noisy, accented, or multilingual audio Whisper large-v3 is more resilient because it was trained on a much larger and noisier corpus and covers ninety nine languages. Match the model to your audio quality and language set rather than to a single leaderboard number.

Can Parakeet do real time streaming captions? Yes, and this is its main advantage. The Token and Duration Transducer emits tokens as audio arrives, so end to end latency is in the low hundreds of milliseconds. Whisper works on thirty second windows and cannot stream in the same way, so its caption latency floor is a second or more even at high real time factor.

Which model is faster on an M5 Max? Parakeet, by a wide margin. On a 64GB M5 Max under MLX it runs at a hundred times real time and beyond, because a transducer does far less sequential work than Whisper's autoregressive decoder. Whisper large-v3 lands around twelve to eighteen times real time and turbo around twenty to thirty times.

Does Parakeet support translation and non European languages? No. Parakeet v3 covers twenty five European languages and transcribes only. Whisper covers ninety nine languages and can translate speech into English as it transcribes. If you need wide language coverage or translation, Whisper is the model.

Can I run transcription and a local language model on the same M5 Max? Yes, and this is where Parakeet's efficiency pays off. A 0.6B transducer at a hundred times real time barely touches the memory bandwidth a local language model needs, leaving headroom in the unified memory pool. A 1.5B Whisper decode competes for that bandwidth directly, so it leaves less room for the reasoning stage that follows.