Granite Speech 4.1: ASR comparison and local setup

The video compares IBM Granite Speech 4.1's three roughly 2-billion-parameter ASR variants and shows code for running speech transcription on a local machine. The speaker frames the choice around accuracy, transcript detail and throughput.

The base model supports punctuation, true casing and keyword biasing for names or technical terms. The speaker reports a leaderboard word error rate of 5.33 and describes multilingual transcription and bidirectional speech translation. Plus adds speaker labels, word-level end timestamps and incremental decoding using previous text as a prefix. Its tradeoffs include five supported languages, no Japanese or translation, and a higher word error rate.

NAR edits a draft transcript through non-autoregressive LLM-based editing. The speaker cites a model-card throughput claim of 1,820 times real time with batching on an H100, but says his own tests did not reach that speed. NAR omits translation, keyword biasing, speaker attribution and timestamps.

The local AI demonstration uses a Dell Pro Max Tower T2 with an RTX Pro 6000 Blackwell GPU and the Transformers library. Setup notes cover matching PyTorch, CUDA and Flash Attention; the speaker compiled Flash Attention for CUDA 13 and warns that older T4 GPUs may cause installation problems. Long-audio chunk sizes and transcript stitching remain areas he is testing.