Ollama and Open WebUI: set up AI on a CPU-only VPS

Set up Llama 3.2 3B with Ollama, Open WebUI and Caddy on Ubuntu, and learn the limits of a VPS with 8 GB RAM and no GPU.

Player not loading? Watch on YouTube

This tutorial builds a self-hosted chat assistant with Ollama, Open WebUI and Caddy in three Docker containers. The host is a sponsored Hostinger VPS with two CPU cores, 8 GB RAM and 100 GB storage.

The setup uses Ubuntu 24.04 and a single Compose file. It covers creating a normal user, adding an SSH key, disabling root and password login, and enabling the firewall. Ollama runs the model, Open WebUI provides the browser chat interface, and Caddy handles HTTPS for the domain. The first registered account becomes the administrator, so the speaker advises creating it immediately and disabling further signups.

Llama 3.2 3B is the chosen model, described as a roughly 2 GB download. This server has no GPU, which limits model choice and response speed. In a Java Spring code review example, the speaker reports a four-second response that identifies email sending inside a database transaction and recommends constructor injection. The description also reports slow long-form responses and an incorrect expansion of REST.

The speaker values knowing that the server is in Frankfurt and controlling where prompts go. This is a small-question coding assistant experiment, with an explicit acknowledgment that larger models remain useful for other work.