Experimently

Experimently is an open-source experimentation and feature-flag platform for teams running their own infrastructure. It is best suited to product and engineering groups that want broad testing methods and keep assignment data in-house. Its breadth is a strong draw, but self-hosting makes it a poor fit for teams seeking a managed service.

Overview

Experimently brings testing, targeting, rollouts, and analysis into one Apache-2.0-licensed platform. The Free Preview costs nothing and includes all features, but it runs on your AWS account rather than as a hosted service. That avoids a subscription charge while leaving deployment and infrastructure management to your team.

The testing menu extends beyond A/B tests to multivariate and split URL testing, sequential testing, CUPED, Bayesian methods, multi-armed bandits, and LLM or model evaluation. That range can support teams with varied experimentation needs; it may be more operational scope than teams seeking only a simple hosted test tool need.

Key features

Experiment and rollout controls

Sticky assignment keeps participants consistently assigned, while staged rollouts let teams release changes gradually. Mutual-exclusion groups and a global holdout help manage overlapping experiments and maintain a baseline. Automatic rollback can respond to error-rate and latency thresholds, which makes the platform relevant to teams connecting experimentation with release safeguards.

Targeting and analysis

Targeting has more than 20 operators, including semantic version comparisons, geo-distance, time windows, JSON paths, and array operations. Analysis combines frequentist and Bayesian methods with sequential mSPRT testing, multi-armed bandits, and segment breakdowns using Bonferroni correction. This is a substantial toolkit for teams that need several ways to target audiences and assess results, though it calls for users prepared to work with a broad set of controls.

SDKs, deployment, and integrations

SDKs cover JavaScript, React, React Native, Edge, Python, Go, Java, Android, iOS, Flutter, Ruby, PHP, .NET, and Elixir; OpenFeature providers are available for JavaScript and Python. The SDKs are not yet distributed through npm, PyPI, or other package registries, so installation follows each SDK's guide rather than the usual registry workflow.

Deployment options include Docker Compose, Kubernetes through a Helm chart, and an AWS CDK app for ECS Fargate, Aurora PostgreSQL, and ElastiCache in your own AWS account. Assignment data stays on your infrastructure, and there is no hosted tier. Documented integrations include Snowflake, BigQuery, Redshift, Databricks, ClickHouse, MySQL, Jira, Salesforce, GitHub, Slack, and email. Security features include HMAC-SHA256 audit logging, role-based access control, scoped API keys, and SAML and OIDC/OAuth2 SSO; the documentation also identifies SOC 2, ISO 27001, and HIPAA compliance features.

Pricing

The Free Preview costs nothing, is self-hosted on your AWS account, and includes all features. The maker also says its build profiles have no price. There is no free trial; the free offering is the preview itself, rather than a time-limited trial. No paid plan or additional tier is needed to access the listed feature set, but the AWS deployment and infrastructure remain part of the self-hosted model.

Platforms

Experimently lists Android, iOS, Linux, macOS, Windows, web, API, and self-hosted support. Its mobile SDK coverage includes Android, iOS, and React Native, alongside web and server-side SDK options.

Who it's for

Choose Experimently if your team can deploy and operate software, wants a broad mix of experimentation and rollout controls, and values keeping assignment data on its own infrastructure. It is less suitable if you need a hosted tier, or if installing SDKs through standard package registries is important to your development workflow.

Pros and cons

• Pros: All features are included in a free self-hosted preview, so the listed testing methods and controls are not gated behind separate plans.
• Pros: Assignment data stays on your infrastructure, which suits organizations that want control over where it resides.
• Pros: Rollout safeguards, detailed targeting operators, and multiple statistical approaches give teams options across experiment design and release management.
• Cons: There is no hosted tier, so deployment and operations fall to the team.
• Cons: SDKs are not on package registries, adding a manual step for teams accustomed to registry-based installation.

Alternatives

GrowthBook is a reasonable alternative for teams that want open-source self-hosting: its listed plan includes one project, unlimited feature flags, unlimited experiments, and unlimited traffic. Confidence by Spotify may suit teams preferring a freemium option with a free plan that specifies 75,000 credits per month, unlimited seats and flags, and up to five surfaces.

DevCycle offers a free plan with stated client and configuration-request quotas, making it an option for teams evaluating a capped freemium service. LaunchDarkly Experimentation has a free Developer plan with 100K experimentation MAU per month, one project, and three environments. Statsig lists a Developer plan with 2M events per month, unlimited flag and config checks, and unlimited seats.

Crazy Egg, Firebase Cloud Messaging, and Omniconvert Explore are also alternatives to consider. Browse the A/B Testing Tools category to compare more options.

Verdict

Experimently is a strong choice for technically equipped teams that want a broad experimentation toolkit, integrated rollout safeguards, and assignment data kept on their own infrastructure. Its all-features free preview lowers the software cost, but it does not remove the work of running the stack. Choose a hosted alternative if you do not want to manage deployment yourself.