Hermes Agent setup with llama.cpp and Gemma 4

This tutorial follows a Windows installation of Hermes Agent, then replaces its connected onboarding model with a local LLM. The speaker uses the desktop app's guided tour and asks Hermes to create an HTML explanation of its own agent loop, tools and skills.

In Settings > Providers > Local Models, Hermes recommends a model based on the computer's hardware. The walkthrough installs llama.cpp through the runtime installer, searches Hugging Face for Gemma 4 and selects a download before starting the runtime and making the model the default. The speaker explains quantization and the need to leave memory headroom for context. His machine has 64 GB of unified memory; the demonstrated chat reports roughly 13 tokens per second. Those figures describe his setup, rather than a general performance guarantee.

The memory example saves a project description and response preferences, then recalls the project in a new session. Later sections cover memory providers, Docker as a terminal execution backend, messaging gateways and separate agent profiles. Obsidian workflows appear as suggestions for further tutorials.

The speaker presents local inference as a way to keep chats private, but explicitly notes that browser use and external tools can still send information outside the computer. The initial guided onboarding also uses a connected trial model; he describes an alternative installation path that skips that connection.