Spring AI 2.0 Tool Search setup and token comparison

This tutorial builds a Spring AI application that lets a model search for relevant tools instead of receiving every tool definition with each request. The speaker compares requests using 47 fictional Initech tools, with custom advisors that log tool availability and token usage.

The example uses Java, Maven, Spring Boot 4.1.0 and JDK 26, with Anthropic as the model provider. The speaker reports that tool search did not work in Spring AI 2.0 for his setup and switches to a 2.0.1 snapshot, adding snapshot repositories and the required dependencies. This is an API-backed example, rather than a demonstration of how to run models locally.

The configuration enables the Tool Search Advisor, selects a vector index and limits search results to three. A simple in-memory vector store and an embedding model support semantic search. The walkthrough also covers retaining discovered tools for subsequent steps and supplying a conversation ID to track the session. Keyword and regular-expression search are other options discussed.

In the reported tests, loading all tool definitions uses 5,141 tokens for a general question and 5,166 for a date question. Tool search answers the date question with 1,166 tokens after discovering and calling the relevant tool. These are demonstration results, not a guaranteed reduction. The speaker recommends considering search for large tool sets; a few compact, frequently used tools may not need it.