Ollama on Proxmox LXC: NVIDIA GPU setup

This CLI tutorial sets up Ollama as a local LLM backend inside a Proxmox LXC. The speaker uses Debian 13, Proxmox VE 9.1.7, Ollama pinned to 0.22.0, and an RTX 3060 with 12GB VRAM. The metadata specifies NVIDIA driver 595.58.03. The guide uses LXC rather than Docker.

Host preparation covers blacklisting nouveau and nvidiafb, rebuilding initramfs, rebooting, and installing the NVIDIA driver with DKMS. The demonstrated privileged container has four CPU cores, 8GB RAM, 2GB swap, and a 20GB root filesystem. These are the speaker's allocations, not stated minimum requirements. Inside the container, the same NVIDIA installer supplies userspace components without kernel modules because LXC shares the host kernel.

Ollama is installed manually from an archive and runs under a dedicated system user through systemd. An override binds it to 0.0.0.0 for access from other containers. The speaker tests a small model and checks GPU activity through service logs, the API, and nvidia-smi; the observed speed applies to that demonstration.

The remaining steps add Pi-hole DNS, a Caddy reverse proxy with Cloudflare SSL, and a Proxmox Backup Server job that excludes model files with .pxarexclude. Restoring that backup requires downloading the models again. Separate Uptime Kuma HTTP and ping checks help distinguish an API failure from a container outage.