LiveBench

What LiveBench is

LiveBench is a free, open-source benchmark for evaluating large language models, designed to limit test contamination and use objective evaluation. Its current benchmark has 23 tasks across seven categories: Reasoning, Coding, Agentic Coding, Mathematics, Data Analysis, Language, and Instruction Following. The benchmark refreshes every six months. Its leaderboard presents overall and category scores, with model rows that expand to show subtasks. Insight views cover quality versus cost, ranked cost, and category profiles; the project defines cost per successful task as (Σ cost ÷ Σ questions ÷ score) × 100 for the selected scope. Questions have verifiable ground-truth answers and can be scored automatically without an LLM judge. To limit possible contamination, the project uses newly released questions and material based on recent datasets, papers, news, and movie synopses. The public project provides benchmark code and links to data on Hugging Face. Its documented Python command-line pipeline generates model answers, scores them, and displays results. It supports OpenAI-compatible endpoints and implemented inference for Anthropic, Cohere, Mistral, Together, and Google models.