Mem0 architecture: agent memory and local model choices

This walkthrough explains persistent AI agent memory through Mem0's architecture. The speaker separates conversation history, which supplies context within a session, from a memory service that retains facts across sessions and can share user preferences among agents.

The architecture described uses a main vector store for memories and metadata, a separate vector store for entities linked to those memories, and SQLite for change history and the latest 10 messages. During ingestion, infer=false embeds messages directly; infer=true uses an LLM to extract structured memories with recent context. The speaker notes that hash deduplication catches only memories with identical wording.

Retrieval combines semantic similarity with BM25 keyword scoring and entity boosts. In the example, a request for 10 memories first retrieves 60 candidates. The later scoring steps reorder that pool and cannot introduce memories outside it. The speaker recommends adding query rewriting in the agent harness rather than assuming the memory pipeline handles it.

For a local LLM setup, the speaker recommends extraction models with 1B to 12B parameters and discusses choosing open source embedding models through Hugging Face and the MTEB leaderboard. This is an architecture explanation with model-selection advice, rather than a step-by-step installation guide.