Ollama and Home Assistant: local voice assistant setup

This tutorial explains the speaker's local AI voice assistant setup in Home Assistant, with Ollama running in a container on Proxmox. An AMD mini PC supplies integrated Radeon graphics. The speaker reports faster responses with the iGPU than with the CPU and chose hardware with an OCuLink port for a possible external GPU upgrade.

The selected model is Qwen 2.5-HA with 7 billion parameters, described as the largest model that remained responsive on this setup. A Modelfile sets temperature to 0.2 to make Home Assistant calls more consistent. The speaker uses an 8K context window and keeps the voice model loaded with a keep-alive value of -1. Changes to instructions or exposed entities can slow the next request while the model reloads its cache.

Whisper handles speech recognition and Piper generates speech, both in local Docker containers. Turning off "prefer local commands" sends every request to the LLM. The speaker limits access to 60 entities and recommends clear names and room assignments for household commands.

Demos cover lights, blinds, a computer, weather, and general questions. A larger context window and longer message history produce a one-second difference in the shown comparison; the speaker reports longer waits in other trials. These settings reflect one person's experiments rather than guaranteed performance on other hardware.