Ollama Grid Search

Ollama Grid Search helps developers compare model responses before choosing a model and prompt for a task. It runs experiments across combinations of models, prompts and inference settings, then presents the results for visual review.

• Model and prompt comparisons: Test the same prompt across models or compare different prompts under similar settings.
• Repeated experiments: Run each combination several times to examine differences between responses, and retry individual generations.
• Response metrics: Review generated text alongside inference time, token counts and generation speed.
• Experiment history: Inspect saved experiments, download them as JSON, or repeat a past experiment with its original or modified parameters.
• Reusable prompts: Keep prompts and examples in a prompt database, with custom system prompts and default inference settings for consistent tests.

The desktop app connects to an Ollama server on your machine or a remote server. It also supports llmman, which provides a compatible Ollama API. The selected server processes your prompts, so tests sent to a remote endpoint leave your machine. Sequential requests and limited concurrency let you control the load on that server.

Ollama Grid Search is open source under the MIT license. Its experiments compare generated responses and inference settings rather than training models.