AgentDojo

What AgentDojo is

AgentDojo is a free, self-hosted environment for researchers evaluating prompt injection attacks and defenses for LLM agents. It installs with pip and provides benchmark suites plus a script for selecting models, attacks, defenses, suites, and tasks. Researchers can extend supplied benchmarks, define new task suites and tasks, or create benchmarks from scratch. Custom attacks can target a pipeline, suite, user task, or injection task. Pipelines combine system messages, LLMs, tool execution, and defense components. Documented tool-calling implementations cover OpenAI, Anthropic, Google, and Cohere, while a prompting interface supports other models; the documentation also describes Llama 3 70B through TogetherAI. Its Python functions runtime runs functions for function-calling models, including stateful environments. Results can be inspected as traces in Invariant Explorer. The project warns results are not a leaderboard because not all combinations were tested. The prompt-injection detector requires the transformers extra, and the package API may change.