understory: shared memory setup with llama.cpp and MCP

understory is an open source memory layer built around Markdown files, Karpathy's LLM Wiki idea and Google's Open Knowledge Format (OKF). The speaker explains how a librarian agent handles memory requests for other tools through MCP. In his setup, it uses the same llama.cpp server as the rest of his local AI stack, without a cloud API in this demonstrated loop. The project also supports external providers such as Anthropic and OpenRouter; local storage alone does not make those configurations offline.

The walkthrough focuses on making memory usable with a local LLM. An initial skills file failed to keep files in the right folder or update indexes reliably. The replacement harness enforces structural rules and leaves decisions about content to the librarian. MCP tools let clients query, update and append memories; a web interface shows memory links and query traces.

Two failures receive particular attention. Client models did not know to search memory, so the speaker added a startup summary through MCP instructions and the query tool description. New facts also became disconnected files. He addressed this by asking the librarian to update existing concepts first and link new concepts in both directions. Validation checks flag orphan concepts and broken links, while replacement rules remove outdated facts.

The speaker reports that Gemma 12B struggled with memory organization, while Qwen 3.6 35B A3B handled it on a single consumer GPU. Recent-memory caching and idle-time consolidation are presented as planned work.