18.7
Vellum describes itself as a personal AI assistant that can read files, manage calendars, send email, run code, and control a computer.
Free planOpen sourceAndroidiOSWeb
Best AI LLM Evaluation Tools in 2026
Assessing language-model quality means looking beyond a single output. AI LLM evaluation tools help organize checks of model responses, prompts, and safety behavior. Compare evaluation methods and model support to see what kinds of assessments are represented, and consider safety evaluations alongside prompt versioning when tracking changes matters to your workflow. Vellum, Weights & Biases, and Evidently AI are…
28.4
Opik helps teams building applications or agents that call an LLM log actions and evaluate behavior through development, testing, and production.
Free planOpen sourceWebAPILinux
38.3
DeepEval is a free, open-source framework for creating evaluation pipelines that test AI systems.
Free planOpen sourceWebAPILinux
- 4
Evidently AI
Evidently AI evaluates, tests, and monitors machine-learning models, LLMs, RAG applications, and AI agents.
Free planOpen sourceWebAPILinuxmacOS8.3From $80 a month - 5
Weights & Biases
Weights & Biases is an AI developer platform for building AI agents, applications, and models.
Free planAndroidiOSWebAPI8.3From $60 a month - 6
Langfuse
Langfuse is an open-source AI engineering platform for teams debugging, analyzing, and iterating on AI agent applications.
Free planOpen sourceWebAPILinuxSelf-hosted8.2From $29 a month - 7
LangWatch
LangWatch is an AI agent engineering platform for teams preparing agents for production.
Free planOpen sourceWebAPILinuxmacOS8.2From €29 a month - 8
LangSmith
LangSmith is a platform for building, evaluating, deploying, monitoring, and improving AI agents.
Free planWebAPILinuxmacOS8.0$39 usage-based - 9
Promptfoo
Promptfoo helps developers and enterprises evaluate prompts and test AI applications for security risks.
Free planOpen sourceWebAPILinuxmacOS7.8Free plan - 10
Braintrust
Braintrust is an observability platform for teams improving AI agents that run in production.
Free planWebAPISelf-hosted7.7From $249 a month - 11
Giskard
Giskard is a freemium platform for finding security and quality problems in AI agents before production use.
Free planOpen sourceWebAPILinuxSelf-hosted7.6Free plan - 12
TruLens
TruLens is a free, open-source Python SDK for tracing and evaluating AI agents, RAG applications, MCP apps, and summarization systems.
Free planOpen sourceAPISelf-hosted7.2Free plan - 13
Maxim AI
Maxim AI is an evaluation and observability platform for simulating, assessing, and monitoring AI agents.
Free planWebAPISelf-hosted7.0From $29 a month - 14
Confident AI
Confident AI is an AI quality platform for evaluating application changes during development and monitoring AI applications in production.
Free planWebAPISelf-hosted6.9From $200 a month - 15
Galileo
Galileo is an observability, evaluation, and guardrail platform for generative AI and agentic applications.
Free planWebAPISelf-hosted6.9From $100 a month - 16
HELM
HELM is a free, open-source Python framework for evaluating foundation models, including large language and multimodal models.
Free planWebAPISelf-hosted6.5Free plan - 17
HoneyHive
HoneyHive is an agent observability platform for teams that want to observe, evaluate, and improve AI agents in production.
Free planWebAPISelf-hosted6.4Free plan - 18
Arize Phoenix
Arize Phoenix helps teams trace, evaluate, debug, and improve AI applications.
Free planWebAPISelf-hosted6.3Free plan - 19
Rhesis AI
Free planWebAPILinuxSelf-hosted6.2Free plan - 20
NVIDIA NeMo Evaluator
NVIDIA NeMo Evaluator is an open-source service for assessing generative AI applications, including language models, RAG pipelines, and agents.
Free planAPILinuxSelf-hosted6.0Free plan - 21
LM Evaluation Harness
LM Evaluation Harness is a unified framework for evaluating generative language models across evaluation tasks.
Free planAPILinuxSelf-hosted5.9Free plan - 22
Inspect AI
Inspect AI is a free, open-source framework for evaluating large language models across coding, agent tasks, reasoning, knowledge, behavior and multimodal understanding.
Free planAPIExtension5.8Free plan - 23
RAGChecker
RAGChecker is a free, Apache-2.0 licensed framework for evaluating and diagnosing retrieval-augmented generation systems.
Free planSelf-hosted5.8Free plan - 24
Ragas
Ragas is a free, self-hosted library for systematically evaluating large language model applications.
APILinuxSelf-hosted5.7 - 25
UpTrain
WebAPISelf-hosted5.7