Pydantic AI and GEPA: prompt optimization tutorial

Samuel Colvin walks through prompt optimization with Pydantic AI, GEPA and Logfire. His example extracts political ancestors from MPs' Wikipedia pages as structured data. The task exposes recurring mistakes: including spouses or children, and treating public figures as politicians.

The workshop starts with environment setup and model API credentials, then compares simple and more detailed prompts across 65 test cases. Colvin reports roughly 87% versus 92% accuracy in that comparison, followed by a 96.7% score after GEPA optimization. These are results from the workshop dataset. Colvin says the golden reference data came from Opus 4.6 and subsequent checking, and acknowledges that it may contain errors. The discussion also covers verbose prompts, variance and overfitting, including an attendee's result that wrongly excluded aunts and uncles.

A locally running FastAPI app demonstrates Logfire managed variables for changing instructions and model settings without redeployment. The demonstration includes restarts and a failed attempt to correct an extraction error. Model calls use external APIs; this is not an offline inference setup. Colvin also discusses implicit user feedback and says enterprise customers can use self-hosted Logfire inside their VPC, or record performance metrics without raw inputs and outputs.