Giskard

Giskard Hub is an AI agent testing platform available hosted or on-premise for organizations that need testing within their own environment. Developers can also run its open-source Python library locally, but the library's LLM judges and attack generators require a model provider and API key, with integrations for OpenAI and Anthropic. Local execution doesn't make those evaluations offline.

The library combines ordinary assertions with model-based assessments across multi-turn conversations. Its scanner generates adversarial scenarios from an agent description and tests for prompt injection, harmful content, stereotypes and misinformation. Developers can add their own test generators. Knowledge-base checks assess whether a RAG application's answers contain unsupported claims.

Hub tests text-based conversational agents through their API, without needing access to the underlying models or vector database. It checks confidential data disclosure as well as answer failures such as contradictions, omissions and incorrect refusals. Domain experts and product managers can contribute test cases through a collaborative red-teaming interface and annotation tools.

Giskard also provides a commercial assessment service that ranks findings by severity, helps prioritize fixes and repeats tests to verify remediation. The Python library uses the Apache 2.0 license, and aggregated analytics are optional. The hosted service offers EU or US data processing; on-premise Hub deployment is available for sensitive workloads.