Garak

Garak is an open-source LLM vulnerability scanner for developers and security teams assessing models or dialogue systems. It tests local models as well as cloud services, so you can assess a model running on your own hardware or an application exposed through an API. The Python tool uses the Apache 2.0 license.

Its probes try to trigger prompt injection, jailbreaks, data leakage, hallucinations, misinformation and toxic output. It combines fixed tests with dynamic and adaptive probes to explore how a model fails. A plugin system supports different model connections, probes and detectors, rather than restricting assessments to one provider or one type of attack.

For local inference, Garak supports Hugging Face Transformers models and GGUF models through llama.cpp. API connections include OpenAI, AWS Bedrock, Replicate, Cohere, Groq and NVIDIA NIM, with LiteLLM support as well. Its REST integration can assess custom endpoints that return plain text or JSON. Local targets run on your hardware; cloud targets receive test prompts through their respective services, and connections such as OpenAI and Replicate require API credentials.

Results show which probes triggered unwanted behavior and the failure rate for each detector. Detailed JSONL logs retain the scan record for closer analysis. This gives teams evidence about specific weaknesses in the model or system they tested, including cases where only some prompt attempts produced a failure.