VRUNAI

What VRUNAI is

VRUNAI is a free, open-source tool for evaluating LLM agents across execution paths, tool calls, and final outcomes. Users define tools, mock data, conditional flows, and test scenarios in YAML without writing code. Each scenario can run repeatedly, allowing consistency scoring for the agent’s path. VRUNAI calculates cost per scenario and provider from token usage and includes model-pricing and context-window statistics. Listed providers include OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Ollama, and custom providers. It offers a terminal interface and a React web app that can run locally or in a hosted browser version. Local inference works through Ollama and custom API endpoints, including OpenAI-compatible APIs. The CLI runs on the user’s machine; web-app keys are stored in browser localStorage and sent directly to configured provider APIs. VRUNAI states that it has no accounts, backend, or telemetry and is fully client-side. The tool has no VRUNAI usage limits or fees, although configured providers may charge for API calls. It is licensed under AGPL-3.0.