Hermes Desktop with a local llama.cpp server

This tutorial connects the Hermes Desktop app at github.com/fathah/hermes-desktop to a local LLM server running llama.cpp with Gemma 4 E4B Instruct Q8 GGUF. The presenter installs the Windows release and points out a macOS DMG. The walkthrough assumes llama.cpp already runs; it refers viewers to a separate installation video.

In the model settings, the presenter selects a local provider, enters the model name, and uses localhost on port 8080 without an API key. He corrects an earlier step: the base URL should omit /v1. His server configuration uses 70,000 tokens of context, GPU layer offloading and flash attention. He recommends roughly 70K context for his agent tasks; this is personal guidance rather than a universal requirement. He reports using an 8GB GPU and 16GB RAM and acknowledges slow responses.

The interface walkthrough covers profiles, skills, persona settings, memories and tool switches. A chat request creates a recurring Telegram message, then changes it to an Ethereum price check every three minutes. Gateway setup uses a bot token and an allowed user ID. Inference runs locally, while Telegram delivery and current price retrieval require network access.

Office mode visualizes agents in a 3D space. The presenter also briefly compares Ollama and LM Studio with llama.cpp, describing LM Studio as easier to integrate and llama.cpp as giving more settings. He prefers local inference for repetitive tasks to avoid unpredictable API costs.