Parakeet, Canary and NVIDIA NIM: local voice agent tutorial

Bruno Capuano and Don explain NVIDIA's speech models and demonstrate a Python workflow for podcast transcription. The tutorial starts with a local audio file, converts it to the model's required format, loads the model through NeMo libraries and generates a transcript. Bruno tests short MP3 clips while monitoring GPU activity.

For his Spanish podcast, Bruno switches from the English Parakeet model he tested to multilingual Canary 1B. The walkthrough explains that NeMo downloads a model on first use and keeps it in a local cache. Initial downloads and model loading add delays; keeping the model in memory avoids repeated loading.

The application then becomes a client/server setup with a browser interface and a FastAPI backend, coordinated through Aspire. A separate NVIDIA NIM container runs Llama 3.2 to suggest podcast titles, descriptions and tags from a transcript. The speakers also explain container deployment through Microsoft Foundry and Azure.

A local AI agent combines speech recognition, TinyLlama and speech output. Its answers include factual errors, and the final live demo lacks shared audio. These limitations matter when assessing the example: it demonstrates how to connect the components, while answer accuracy still needs attention. GitHub Copilot helps Bruno research model choices and plan the transcription script.