LM Evaluation Harness
5.9#16 in llm evaluation toolsFree planFree planAPILinuxSelf-hosted
Visit
| Fact | LM Evaluation Harness | Promptfoo |
|---|---|---|
| Installation | The base package provides the core framework while model backends are installed separately as optional extras. | Installation is available through npm, npx, Homebrew, and a project dependency; npm and npx require Node.js 22.22.0 or newer. |
It suits researchers, organizations, and model developers who need to evaluate language models across established or custom tasks. It may also fit teams seeking a self-hosted evaluation framework with multiple model and API options.
Promptfoo suits developers and enterprise teams evaluating prompts, model outputs, and AI application risks. Its local and self-hosted options may appeal to teams that want to run the Community plan on their own infrastructure.