Promptfoo
7.8#2 in llm evaluation toolsFree planFree planWebAPILinuxmacOS
VisitPromptfoo vs Rogue
What both publish
| Fact | Promptfoo | Rogue |
|---|---|---|
| Evaluation | It tests prompts across multiple providers, grades outputs with assertions, and calculates accuracy, safety, and performance metrics. | Automatic Evaluation tests agents against business policies and expected behaviors with pass/fail reports and reasoning. |
| License | The open-source project is distributed under the MIT license. | The repository license is the Qualifire OSS License, combining MIT terms with a Commons Clause that excludes selling the software or offering it as a paid hosted service. |
| Model providers | Provider documentation covers OpenAI, Anthropic, Google, Vertex, HTTP, JavaScript, Python, Ruby, shell, OpenAI Agents, and other services. | Rogue lists OpenAI, Anthropic, and Google models via LiteLLM and requires an LLM API key to get started. |
| Product | Promptfoo helps developers and enterprises build secure, reliable AI applications. | Rogue is an AI agent evaluation and red teaming platform for testing agent reliability and security. |
| Red teaming | Its red-team workflow generates adversarial tests for vulnerabilities before deployment and maps testing to OWASP and NIST frameworks. | Red Teaming simulates adversarial attacks to find security vulnerabilities. |
Promptfoo plans
- CommunityFree
- EnterprisePriced on request
- On-PremisePriced on request
Rogue plans
- Personal and internal useFree
- Commercial hostingPriced on request
Pick Promptfoo if…
Promptfoo suits developers and enterprise teams evaluating prompts, model outputs, and AI application risks. Its local and self-hosted options may appeal to teams that want to run the Community plan on their own infrastructure.
Pick Rogue if…
Rogue is aimed at security teams and developers building or deploying AI agents. It may suit teams that need regression testing, security audits, or compliance reporting.