Giskard: a guide to testing AI agents and RAG

Giskard co-founder Alexandre Combessie explains how his team tests AI agents when several different answers can be valid. He describes LLM judges that check contradictions, omissions and added information, with concise explanations so people can inspect failed tests. In a RAG application, information absent from the supplied knowledge base gets flagged, but a domain expert still needs to decide whether it is false or inappropriate.

An HR assistant provides the example for a two-stage workflow: an automated scan probes security and business risks, then teams build more specific tests using their own data. Combessie says scans take roughly five to eight hours. Hallucination checks require the company's knowledge base alongside the agent endpoint. Scan reports include tests that teams can add to CI/CD for regression coverage; he recommends running them with each code change.

These company-platform services are discussed alongside the open-source library. The discussion separates preproduction evaluation from runtime safeguards. Combessie describes customizable guards that can block specific risky behavior or log it, and says they must sit near model inference in the client's infrastructure. He presents the guard module as a beta for existing customers, with broader availability tentatively planned for early 2027. For QA practitioners, he recommends Giskard's open source version as a starting point for generating tests and integrating them into CI.