FlakyTest.dev
FlakyTest.dev is a web tool for Python teams managing and monitoring test suite performance.
What FlakyTest.dev is
FlakyTest.dev is a web tool for Python teams managing and monitoring test suite performance. Its pytest plugin tracks tests and supports test management, while ownership details help direct flaky-test notifications to the person who wrote the test. Users can mute a test in the interface without code changes or a pull request; it keeps running and logging errors but no longer fails the CI build. The service shows every run for an individual test, including standard output and standard error for failures. Run statistics can help identify tests that slow builds, and flake detection and a quarantine workflow support handling unreliable tests. The setup guide documents GitHub Actions configuration using the FLAKYTEST_SECRET_TOKEN repository secret; the plugin sends results when that environment variable is set. The OSS plan is free for personal projects, nonprofits, and open source projects. The Business plan has a 60-day free trial, with pricing afterward to be determined. The product currently focuses on Python, with other languages described as a future goal.
Good
- Routes flaky-test notices using test ownership
- Mute tests without code changes or pull requests
- Shows run history and failure output
- Offers flake detection and a quarantine workflow
Watch out
- Currently focused on Python projects
- Business pricing after the 60-day trial is undetermined
Who it suits
FlakyTest.dev suits Python teams dealing with unreliable test suites, particularly those who want test ownership, run history, and ways to mute tests without changing code. Its current language focus is Python.
Verdict
FlakyTest.dev combines test history, debugging details, and CI-friendly muting for Python projects. Teams should account for its current language focus and determine Business pricing after the trial.
Plans and prices
From FlakyTest.dev's own pricing page, checked 4 Oct 2026.
| Plan | Price | What it gives you |
|---|---|---|
| OSS | Free | Personal projects · non profits · open source projects · Free for personal projects, non profits, and open source projects. |
| Business | Priced on request | 60 day free trial · pricing after trial to be determined · Free 60 day trial, help me figure out pricing after that |
The facts
Each one read from FlakyTest.dev's own pages, with the date it was checked.
What it does
- Data sent
- The pytest plugin sends test results to FlakyTest when the FLAKYTEST_SECRET_TOKEN environment variable is set.checked 4 Oct 2026
- Debug details
- For failing tests, it shows debug information including standard output and standard error.checked 4 Oct 2026
- Integrations
- The setup guide documents GitHub Actions configuration using a repository secret named FLAKYTEST_SECRET_TOKEN.checked 4 Oct 2026
- Intended users
- The maker describes FlakyTest as a tool for Python teams dealing with unreliable test suites.checked 4 Oct 2026
- Language limit
- The maker says the product currently focuses on Python projects and plans to support other languages in the future.checked 4 Oct 2026
- Mute tests
- Users can mute a test in the UI without code changes or pull requests, while the test continues running and logging errors without failing the CI build.checked 4 Oct 2026
- Purpose
- FlakyTest is a place to manage, monitor, and report on test suite performance.checked 4 Oct 2026
- Run statistics
- Run statistics can help identify tests that make builds slow.checked 4 Oct 2026
- Security and privacy
- The privacy policy says the service uses commercially acceptable means to protect personal data but cannot guarantee absolute security.checked 4 Oct 2026
- Support
- The documentation directs users needing help, support, or feedback to email the maker.checked 4 Oct 2026
- Test framework
- Its pytest plugin enables test tracking and management for Python projects.checked 4 Oct 2026
- Test history
- The product shows all runs for an individual test.checked 4 Oct 2026
The company
- Headquarters
- Ljubljana, Sloveniachecked 28 Sep 2026
- Maker background
- Anže describes himself as a software engineer and indie maker with over ten years of experience shipping products for Silicon Valley startups and growing development teams.checked 4 Oct 2026
- Test ownership
- It uses test ownership to direct flaky test notifications to the person who wrote the test.checked 4 Oct 2026
Alternatives to FlakyTest.dev
- 1
Cypress
Free planOpen sourceWebAPILinuxmacOS8.4From $67 a month - 2
Qualflare
Free planAndroidiOSWebAPI8.2From $16 a month - 3
TeamCity
Free planAndroidWebAPILinux8.2From $15 a month - 4
Captain
Free planOpen sourceWebAPILinuxmacOS8.1$10 one-time - 5
Bitbucket Pipelines
Free planWebAPILinuxmacOS8.0From $3.65 a month - 6
BuildPulse
Free planWebAPILinuxmacOS7.9Free plan