Odysseus and llama.cpp: a local agent setup overview

This overview uses PewDiePie's Odysseus project to explain how an AI agent works and how a smaller model could power it. The presenter outlines installation through a cloned GitHub repository and Docker, then tours model selection, memory, skills and document tools. He also reports problems: a hardware recommendation includes an embedding model, and a greeting triggers an unnecessary web search.

The central distinction is between a self-hosted application and a local LLM. Odysseus can run on a computer while sending model requests to a cloud provider. The demonstration uses Mistral Large 4 through OpenRouter. The presenter then introduces llama.cpp and the Llama macOS app to run models locally, with Hugging Face hardware settings helping select model sizes and quantization. His example computer is an Apple M4 Max with 36 GB of memory; Q4 and BF16 recommendations depend on the selected model.

The final section discusses Ajax, described as a fine-tuned Qwen 3.5 9B model. The presenter proposes training on successful tool-call conversations, either collected from the same harness or sourced from public datasets. Much of his account of PewDiePie's training process is explicitly speculative. This is a conceptual guide rather than a complete training walkthrough. It points to further tutorials and advises cleaning traces, removing credentials and comparing model behavior before and after fine-tuning.