Sensei

What it does

Agent connection

The site shows agents connecting to Sensei over HTTP or stdio.checked 4 Oct 2026

Built-in suites

The site lists built-in suites for SDR, customer support, content writing, bartender, dungeon master, and cat interview roles.checked 9 Oct 2026

CI/CD

The repository lists CI/CD integration among Sensei's capabilities.checked 9 Oct 2026

Community library

The home page describes a library of 70+ community-built professional evaluation suites.checked 4 Oct 2026

Custom suites

Users can create their own evaluation suites and publish them to the community marketplace.checked 9 Oct 2026

Evaluation

It evaluates agents across task execution, reasoning, and self-improvement.checked 4 Oct 2026

Evaluation layers

Sensei evaluates task execution, reasoning, and self-improvement through feedback.checked 9 Oct 2026

Integrations

The repository documents HTTP, stdio, OpenAI-compatible, and LangServe adapters for connecting agents.checked 9 Oct 2026

Intended users

Sensei is designed for people evaluating AI agents on real-world professional tasks using standardized benchmarks, scenarios, and measurable KPIs.checked 9 Oct 2026

License

Sensei is distributed under the MIT License.checked 9 Oct 2026

LLM judging

Sensei supports LLM-based and comparative judging, with OpenAI shown as an example provider.checked 9 Oct 2026

Marketplace

The marketplace lets users discover, download, and publish AI-agent evaluation suites.checked 9 Oct 2026

Marketplace categories

Marketplace categories include engineering, sales, marketing, product, design, support, testing, analytics, compliance, and content.checked 4 Oct 2026

Purpose

Sensei is an open-source qualification engine for testing, evaluating, and certifying AI agents across professional skills.checked 9 Oct 2026

Reports

The repository documents machine-readable JSON results and terminal reports.checked 9 Oct 2026

Scoring

Its evaluation pipeline produces a score from 0 to 100 and a badge decision.checked 9 Oct 2026

Security policy

The security policy supports version 0.x.x and asks vulnerability reporters to contact the maintainer by email rather than opening a public issue.checked 9 Oct 2026

Security response

The policy says reports are acknowledged within 48 hours, initially assessed within one week, and targeted for a patch within 30 days of confirmation.checked 9 Oct 2026

Test suites

Built-in suites cover roles including SDR, customer support, content writer, bartender, dungeon master, and cat interview.checked 4 Oct 2026