PyRIT overview: AI red teaming architecture and scoring

Roman Lutz introduces PyRIT (Python Risk Identification Toolkit), an open source framework for automated and human-guided testing of generative AI systems. The talk explains architecture for assessing systems the tester is authorized to examine. It does not demonstrate installation or an actual test run.

Lutz explains a workflow that combines attack prompts, converters, target endpoints and response scoring. Converters change inputs through translation, encoding or transformations across text and images. He describes support for OpenAI-compatible endpoints, including Ollama, which makes the toolkit relevant to testing a local LLM. Playwright provides a browser automation route for applications without a suitable API, though Lutz notes that browser tests can be flaky. Results can go into a local SQLite database or a shared Azure SQL database.

The discussion then moves to iterative attacks that start with an objective and adapt to the target's replies. These can include backtracking and branching. Lutz says access to a sufficiently capable adversarial model is a practical obstacle, while endpoint quota and throughput constrain parallel testing.

Human review remains part of the process. Lutz recommends comparing automated scores with human labels before scaling up, since false positives and missed harmful responses affect the results. He also advises using a sandbox when testing an AI agent that can operate a computer.