IBM's 470M Speech Model Transcribes 3.5 Hours Per Second

Granite Speech 5.0 TurboCTC ditches the language model and hits over 12,600 RTFx — making it one of the fastest speech recognisers available.

By The ai-compute Desk

IBM's Granite Speech 5.0 TurboCTC can transcribe more than 3.5 hours of speech in one second using batched inference on an NVIDIA H200 GPU — a throughput figure of over 12,600 RTFx. [1] Two variants launched simultaneously: an Apache 2.0-licensed model and a non-commercial CC-BY-NC-SA-4.0 edition trained on additional data. [1]

The speed comes from a structural decision: these are encoder-only models, unlike prior Granite Speech models which comprise an acoustic encoder, projector, and Granite LM with LoRA adapters. [1] Dropping the language model delivers over 20x faster throughput than previous Granite Speech models while keeping the parameter count at 470M. [1]

Accuracy holds up at that pace. The non-commercial model scores an aggregate 4.85% word error rate on the OpenASR Leaderboard public test sets; the Apache 2.0 model scores 5.00% WER. [1] On the FFASR far-field leaderboard as of 25 August 2026, the Apache 2.0 model ranked ninth in accuracy and the non-commercial variant ranked fifth, while both models were the fastest two on the board. [1]

The throughput gain traces to token-rate reduction. 0 models generate 12.5 tokens per second, achieved through three stages of 2x subsampling — the second and third built into the first two Conformer blocks in the stack. [1] The trade-off is capability: speech translation and keyword biasing, features of the LM-equipped predecessors, are gone. The models are ideal for speech-to-text tasks on edge devices. [1]

Speed and accuracy have long pulled in opposite directions in speech recognition. What makes these models worth watching is how little accuracy moves when the throughput lever is pushed this far.

Granite Speech 5.0 TurboCTC is supported natively in transformers, with both model weights available on Hugging Face. [1]

they can transcribe more than 3.5 hours of speech in one second using batched inferenceIBM Granite team