PromptLens
PromptLens is a web platform for teams managing prompts, evaluating changes, and observing production LLM workflows.
What PromptLens is
PromptLens is a web platform for teams managing prompts, evaluating changes, and observing production LLM workflows. Teams can compare a candidate prompt with the production version on the same dataset, then inspect score changes, generated outputs, and failures. An LLM judge checks outputs against expected answers and reports pass or fail with a brief explanation. Users can version prompts and publish the needed version to an application through a retrieval API. Its observability product traces production model calls, tools, and retrieval, including timing, errors, usage, and cost. The homepage lists providers such as OpenAI, Anthropic, Google, xAI, Meta, DeepSeek, Mistral, Cohere, and Amazon. Organizations may connect a provider API key or use purchased OpenRouter credits without adding a key. Shared reports open from a link without an account or paid seat. Evaluations provide evidence and warnings for release decisions, but do not automatically gate releases. Product access is subject to early-access approval. The Free plan is available; Pro costs $99 per organization per month.
Good
- Compare prompt versions on the same dataset.
- LLM judge returns pass or fail with explanations.
- Traces model calls, tools, and retrieval.
- Shared reports need no account or paid seat.
- Free plan is available.
Watch out
- Product access requires early-access approval.
- Evaluations are not an automatic release gate.
- Pro costs $99 per organization per month.
Who it suits
PromptLens suits teams shipping production LLM workflows that need prompt versioning, evaluation, or observability. It may also fit teams that want to share reports with people who do not have an account or paid seat.
Verdict
PromptLens brings prompt management, evaluation, and production tracing together for LLM teams. Its release evaluations inform decisions rather than enforcing them, and access is subject to early-access approval.
Plans and prices
From PromptLens's own pricing page, checked 2 Oct 2001.
| Plan | Price | What it gives you |
|---|---|---|
| Free | Free | 2 members · 3 projects · 3 prompts · up to 10 test cases per dataset · 50 evaluation runs per member per 24 hours · 50 playground calls per hour · $0 per organization / month |
| Pro | $99 a month | Unlimited members · 10 projects · 10 prompts · up to 500 test cases per dataset · 100 evaluation runs per member per 24 hours · 500 playground calls per hour · $50 welcome model credits that never expire · priority email support · $99 per organization / month |
| Teams | Priced on request | Custom project, prompt, and dataset limits · higher evaluation and playground volume · provider policy controls · security review · dedicated support · Custom volume pricing |
The facts
Each one read from PromptLens's own pages, with the date it was checked.
What it does
- Availability
- The evaluations page says product access is subject to early-access approval.checked 2 Oct 2001
- Cookies
- The privacy policy says PromptLens uses essential cookies for sessions and preferences and does not use tracking or advertising cookies.checked 2 Oct 2001
- Data retention
- Users can delete datasets, prompts, and experiments at any time; after account deletion, PromptLens says it will delete the user's data within 30 days.checked 2 Oct 2001
- Evaluation workflow
- Teams can compare a candidate prompt version with the production version on the same dataset, then review score changes, outputs, and failures.checked 2 Oct 2001
- Model access
- Organizations can connect a provider API key or use purchased credits to run models through OpenRouter without adding a key.checked 2 Oct 2001
- Model providers
- The homepage lists OpenAI, Anthropic, Google, xAI, Meta, DeepSeek, Mistral, Cohere, and Amazon among the model providers teams can test across.checked 2 Oct 2001
- Observability
- The site says its observability product traces production model calls, tools, and retrieval, including timing, errors, usage, and cost.checked 2 Oct 2001
- Prompt management
- PromptLens lets users version prompts and publish the version their application needs through a retrieval API.checked 2 Oct 2001
- Release control
- Evaluations provide evidence and warnings for release decisions, but they are not an automatic CI release gate.checked 2 Oct 2001
- Scoring
- An LLM judge compares each output to an expected answer and returns pass or fail with a short explanation.checked 2 Oct 2001
- Security
- The privacy policy says sensitive data is encrypted at rest, data is transmitted over HTTPS, and access to user data is restricted to authorized personnel.checked 2 Oct 2001
- Shared reports
- Anyone with a shared report link can open it without an account or paid seat, and shared report links are unlimited on every plan.checked 2 Oct 2001
- Support
- The Free plan includes email support, Pro includes priority email support, and Teams includes dedicated support.checked 2 Oct 2001
- What it does
- PromptLens provides prompt management, evaluations, and observability for teams shipping production LLM workflows.checked 2 Oct 2001
Alternatives to PromptLens
- 1
Vellum
Free planOpen sourceAndroidiOSWebAPI8.7From $30 a month - 2
Langfuse
Free planOpen sourceWebAPILinuxSelf-hosted8.1From $29 a month - 3
PromptHelm
Free planAndroidiOSWebAPI8.0From $39 a month - 4
Agenta
Free planOpen sourceWebAPISelf-hosted7.9From $29 a month - 5
Promptfoo
Free planOpen sourceWebAPILinuxmacOS7.8Free plan - 6
Braintrust
Free planWebAPISelf-hosted7.6From $249 a month