FlakyWatch
FlakyWatch monitors GitHub Actions test results to identify unstable tests, estimate their CI and developer time costs, and track issues for them.
What FlakyWatch is
FlakyWatch monitors GitHub Actions test results to identify unstable tests, estimate their CI and developer time costs, and track issues for them. Users upload JUnit XML results through a workflow step and the FlakyWatch GitHub App; the service says no agents or config files are required. It supports pytest, Jest, JUnit, RSpec, and other JUnit-compatible runners, classifying tests as stable, flaky, broken, or newly unstable with confidence scores. Its dashboard reports blocked runs, major failures, affected branches, and cost per test. Flaky tests can become GitHub issues with impact and suggested next steps, and issues can be reopened if a flake returns. The free plan supports one repository and 14 days of history; Starter costs $29 a month for five repositories and 30-day history, while Team costs $79 a month for 20 repositories and 90-day history. Paid-plan statistics are retained for 90 days. FlakyWatch also offers an optional weekly email digest; its Slack integration is marked as coming soon on Team.
Good
- Classifies tests with confidence scores.
- Creates GitHub issues for flaky tests.
- Reports blocked runs and cost per test.
- Free plan includes one repository.
- Supports pytest, Jest, JUnit, and RSpec.
Watch out
- Free plan history is limited to 14 days.
- Free plan is limited to one repository.
- Slack integration is marked coming soon.
- Monthly subscriptions are non-refundable.
Who it suits
FlakyWatch suits teams using GitHub Actions that want to classify flaky tests, see their impact, and track follow-up in GitHub issues. It supports several JUnit-compatible test runners.
Verdict
FlakyWatch brings test classification, impact reporting, and GitHub issue tracking together for GitHub Actions results. Check repository limits and history retention against the plan details before choosing.
Plans and prices
From FlakyWatch's own pricing page, checked 3 Oct 2026.
| Plan | Price | What it gives you |
|---|---|---|
| Free | Free | 1 repository · Classification + cost tracking · GitHub issue creation · 14-day history · Weekly digest · forever |
| Starter | $29 a month | 5 repositories · Auto issue creation + lifecycle · Owner suggestions · 30-day history · Email support · monthly |
| Team | $79 a month | 20 repositories · 90-day history · Quarantine rules · Slack integration (coming soon) · CSV export (coming soon) · Priority support · monthly |
The facts
Each one read from FlakyWatch's own pages, with the date it was checked.
What it does
- Classification
- Tests are classified as stable, flaky, broken, or newly unstable, with confidence scores.checked 3 Oct 2026
- Data access
- The privacy policy says FlakyWatch accesses repository metadata, GitHub Actions workflow run data, uploaded JUnit XML artifacts, commit SHAs, branch names, and run timestamps, but not source code or other repository files.checked 3 Oct 2026
- Data retention
- Test execution data is retained for 30 days by default; aggregated statistics are retained for 90 days on paid plans and 14 days on the free plan.checked 3 Oct 2026
- Integrations
- The site lists GitHub Actions and GitHub issues; Slack integration is marked coming soon on the Team plan.checked 3 Oct 2026
- Issue tracking
- Flaky tests can become GitHub issues with classification, impact, and recommended next steps; issues can be reopened if the flake returns.checked 3 Oct 2026
- Reporting
- The dashboard includes impact metrics such as blocked runs, main failures, affected branches, and cost per test.checked 3 Oct 2026
- Security
- The privacy policy states that data is encrypted in transit and at rest, production systems have access controls, and the service uses least-privilege access and regular security audits.checked 3 Oct 2026
- Service limits
- The terms state that the free plan is limited to one repository and 14-day data history, and that monthly subscriptions are non-refundable.checked 3 Oct 2026
- Setup
- The site says setup uses the FlakyWatch GitHub App and a workflow step to upload JUnit XML artifacts, with no agents or config files required.checked 3 Oct 2026
- Support
- FlakyWatch lists email support on Starter, priority support on Team, and says it aims to respond to inquiries within 48 hours during business days.checked 3 Oct 2026
- Test results
- Users upload JUnit XML results from GitHub Actions; the site names pytest, Jest, JUnit, RSpec, and any JUnit-compatible runner as supported.checked 3 Oct 2026
- Weekly digest
- FlakyWatch offers an opt-in weekly email digest with new unstable tests, cost trends, and open issues.checked 3 Oct 2026
- What it does
- FlakyWatch monitors GitHub Actions test results, detects and classifies flaky tests, estimates CI and developer time costs, and tracks them as GitHub issues.checked 3 Oct 2026
Alternatives to FlakyWatch
- 1
Cypress
Free planOpen sourceWebAPILinuxmacOS8.4From $67 a month - 2
Qualflare
Free planAndroidiOSWebAPI8.2From $16 a month - 3
TeamCity
Free planAndroidWebAPILinux8.2From $15 a month - 4
Captain
Free planOpen sourceWebAPILinuxmacOS8.1$10 one-time - 5
Bitbucket Pipelines
Free planWebAPILinuxmacOS8.0From $3.65 a month - 6
BuildPulse
Free planWebAPILinuxmacOS7.9Free plan