Ollama Grid Search
A desktop local LLM evaluation tool for macOS, Windows and Linux. Compare models through local or remote Ollama servers. Open source under MIT.
Ollama Grid Search is a desktop app for comparing LLM responses across models, prompts and inference settings. It runs on macOS, Windows and Linux, and suits developers or anyone choosing a model and prompt combination for a particular task. You can inspect the responses together rather than repeat each test by hand.
The app connects to an Ollama server on your own machine or a remote server. It also works with llmman, a local model runner that provides the Ollama API. The selected server runs the models, so a remote connection sends your test prompts there; the desktop app handles the experiments and result views.
An experiment can test several models against the same prompt, compare different prompts under similar settings, or vary inference parameters across combinations. Repeated runs let you examine how responses differ between attempts. You can compare timing, token counts and generation speed alongside the text, and retry individual responses when needed.
Saved experiment logs make it possible to revisit results, repeat a previous test, or change its parameters for another comparison. You can download experiments as JSON. A prompt database stores reusable prompts and examples, while custom system prompts and default parameters help keep tests consistent.
Ollama Grid Search is open source under the MIT license. Its comparisons focus on generated responses and inference settings, rather than model training. It supports sequential requests or limited concurrency to control the load on the model server.
