Memoturn

Memoturn is an open-source platform for observing and managing LLM applications. It is best suited to teams that want tracing, evaluation, prompt workflows, and monitoring while keeping telemetry on infrastructure they control. Its breadth is useful, but operating the platform and accounting for uneven SDK-language coverage are part of the decision.

Overview

Memoturn combines LLM tracing, evaluations, prompt management, and operational metrics in an Apache-2.0 project. Self-hosting keeps telemetry on infrastructure the operator owns, a meaningful advantage for teams with data-location requirements. The corresponding cost is operational responsibility: deployments can run locally, with Docker Compose, or on Kubernetes through Helm, and teams must choose a telemetry engine. Apache Doris and PostgreSQL are supported, with PostgreSQL suited to smaller installations.

Key features

Tracing and evaluation

Traces, spans, generations, and scores appear on a waterfall timeline, alongside sessions and per-user rollups. This gives teams a way to relate model activity to a larger interaction instead of treating each generation as an isolated event. Evaluation covers offline dataset experiments, production-trace sampling, and human review queues with one-click scoring, so teams can assess behavior before release and monitor it in operation.

Prompts and monitoring

The prompt registry versions prompts across production, staging, and custom channels, and supports A/B experiments that split live traffic between versions. Dashboards report cost, token use, and latency. Alert rules and cost budgets are evaluated every minute; event rules can send Slack messages or webhooks when scores fall or spending spikes. That combination suits teams that want prompt changes and operational signals in the same workflow, though the platform still needs to be deployed and maintained by its operator.

Integrations and data

Memoturn accepts OpenTelemetry data through OTLP/JSON with GenAI semantic conventions, and provides a REST API plus TypeScript, Python, and Go SDKs. Its integration guide includes wrappers for services and frameworks such as OpenAI, Anthropic, Bedrock, Gemini, Groq, Mistral, Cohere, and LangChain, as well as other frameworks and vector stores. Coverage is not uniform: some first-party wrappers are TypeScript-only or Python-only, while Go support includes OTel export helpers and manual tracing. Go-oriented teams should check that distinction against their instrumentation needs before committing.

Governance and agent IDEs

The Apache-2.0 core includes OIDC and SAML single sign-on, organization roles, audit logs, ingest-time PII masking, retention policies, scheduled NDJSON exports, API-key management, and per-project rate limits. These controls make the self-hosted option more relevant to organizations with access and data-handling requirements. An MCP server exposes prompts, datasets, and review queues to agent IDEs such as Claude Code and Cursor, locally or over Streamable HTTP.

Pricing

Memoturn (Apache-2.0 self-hosted core): free. The free plan is open source, and no paid tier or usage limits are stated. It is a strong fit for teams willing to self-host and manage deployment and storage themselves. Its price does not remove that operational work, and language-specific wrapper gaps may require alternate instrumentation approaches.

Platforms

Memoturn supports API, Linux, web, and self-hosted use. Deployment choices include local operation, Docker Compose, and Helm on Kubernetes. Its telemetry can be stored in Apache Doris or PostgreSQL, with the latter described for smaller installs.

Who it's for

Memoturn is aimed at teams building LLM applications that need to debug production traffic, track spend, catch quality regressions, and manage prompts. It is especially appropriate when keeping telemetry on owned infrastructure matters and the team can operate a self-hosted service. Teams seeking a managed service, or those relying on first-party wrappers in a language with narrower coverage, should compare alternatives before choosing it.

Pros and cons

• Pros: Apache-2.0 licensing and self-hosting keep telemetry on infrastructure the operator owns.
• Pros: Tracing, offline and online evaluations, human review, prompt experiments, and cost and latency monitoring span the LLM workflow.
• Pros: Built-in SSO, roles, audit logs, PII masking, retention controls, exports, and API-key management address governance needs.
• Cons: Self-hosting requires the team to deploy and operate the platform and select a supported telemetry engine.
• Cons: Some first-party integration wrappers are limited to TypeScript or Python; Go coverage relies on OTel helpers and manual tracing.

Alternatives

Choose Evidently AI if you want a free, open-source framework with Linux, macOS, Windows, API, web, and self-hosted platforms. W&B Weave is worth considering when a freemium option with cloud plans is preferable; its Free plan includes 1 GB/mo Weave data ingestion, 5 GB/mo storage, and AI application evaluations, tracing, and scorers. Pick OpenLIT if unlimited self-hosted usage, users, and projects on its free OSS plan are the priority.

Opik offers an open-source core for observability and evaluation that users can run locally, making it another option for teams prioritizing downloadable self-managed software. Deepchecks may suit smaller projects that fit its Basic plan: up to 3 seats, 1 AI application, up to 5K DPUs/month, three months of retention, and unlimited prompt-based metrics. LangWatch is an Apache-2.0 open-source alternative whose Developer plan provides 50k events/month and 14-day data access.

NannyML is another option for self-managed open-source use, with a Starter plan at $399 a month for 2 models and 10 M predictions. Qualdo-MQX may fit teams seeking a paid Starter plan, priced at $79 a month for up to 100 GB of data and 10 ML models. Browse Model Monitoring Software for more options in the category.

Verdict

Choose Memoturn if your team needs an open-source LLM engineering platform that combines tracing, evaluations, prompt management, and monitoring while keeping telemetry on infrastructure you own. Its broad workflow and governance controls are the main reasons to choose it. Look elsewhere if you need to avoid self-hosting or depend on complete first-party wrapper coverage in a particular language.