Prompt flow

Prompt flow is a Python toolkit for developers who want to run and evaluate LLM application workflows in their own development environment. Its chatbot examples call OpenAI or Azure OpenAI remotely using API credentials. Workflow execution and evaluation can run locally, while those model requests leave the machine.

Flows connect prompts and model calls with Python code and other tools. Developers can use a visual designer in VS Code alongside the Python SDK and command-line interface. Tracing exposes model interactions so teams can investigate problems across a workflow.

Dataset evaluation lets teams test application behavior across larger collections of inputs and calculate quality and performance metrics. Testing and evaluation can also become part of CI/CD, giving developers a repeatable way to check workflows before deployment.

Flows can run on a serving platform the team chooses or become part of an existing application's code. Azure AI provides a separate cloud offering for team collaboration; developers don't need that offering for local workflow development. Examples include a PDF chat application with evaluation metrics.

The toolkit is open source under MIT. Telemetry is enabled by default and can send usage information to Microsoft. Users can disable that collection.