LM Studio with AnythingLLM: context and agent setup

This tutorial connects LM Studio models to AnythingLLM for document questions and file access. The speaker starts with a model that supports tool calling, identified by a blue hammer icon, then starts the server in LM Studio's developer section. In AnythingLLM, they select LM Studio under Settings > AI providers > LLM and save the connection. The default address uses localhost on port 1234.

The demonstration uses Granite 4.2 with 3 billion parameters. To increase its context window, the speaker ejects the model and reloads it with manually chosen load parameters. They set a context length of 131,000 and report higher memory use. The tutorial places this setting in LM Studio rather than AnythingLLM and cautions that available RAM or GPU memory limits usable context. A PDF question illustrates document summarization.

For the AI agent example, a request to count files initially fails because file system access is disabled. The speaker enables it under Agent skills, grants permission to a specific folder, and starts a new chat. The model first reports 158 items, then returns 159 after a request to check again.

The speaker recommends LM Studio Bionic as a simpler single-app option for local file work. They describe file access, code and document creation, plus a local model API server for other programs.