Cosmic Ray
Cosmic Ray is a free mutation-testing tool for Python 3 projects.
What Cosmic Ray is
Cosmic Ray is a free mutation-testing tool for Python 3 projects. It changes source code in small ways and runs a test suite for each change, checking whether the tests catch the altered behavior. This offers a different view from code coverage, which indicates whether a line ran. Cosmic Ray supports unittest, pytest, and nose. A TOML file specifies the modules to mutate, test command, timeout, exclusions, and distributor. The local distributor runs tests serially; the HTTP distributor sends work to local or remote workers for parallel execution, with each worker needing its own copy of the code. Reports are available through cr-report and cr-html, and users can add distributor or mutation-operator plugins. The tool changes code directly on disk, so its documentation recommends committing changes beforehand. Runs can take a long time because the suite is executed for each mutation, and the full configuration options are not currently well documented. Tests sharing a module with production code may also be mutated unless separated or excluded.
Good
- Supports unittest, pytest, and nose.
- HTTP workers can run mutations in parallel.
- Includes session summaries and HTML reports.
- Custom mutation operators can be added through plugins.
Watch out
- Mutation runs can take a long time.
- Mutations are applied directly to code on disk.
- Full configuration options are not well documented.
- Test code sharing production modules may be mutated.
Who it suits
Cosmic Ray suits Python teams who want to check whether their tests detect behavioral changes, rather than relying only on line execution coverage. It is particularly relevant where teams can configure and run potentially lengthy test suites.
Verdict
Cosmic Ray provides a configurable way to assess whether Python tests catch changed behavior, with local and distributed execution options. Its direct disk changes, runtime demands, and documentation gap call for care before running it.
The facts
Each one read from Cosmic Ray's own pages, with the date it was checked.
What it does
- Code handling
- Cosmic Ray applies mutations directly to code on disk, and the documentation advises committing changes first because a mutation might not be correctly undone after an exception or crash.checked 3 Oct 2026
- Configuration limitation
- The documentation says the full set of configuration options is not currently well documented.checked 3 Oct 2026
- Custom extensions
- Users can provide their own distributor and mutation operator plugins.checked 3 Oct 2026
- Execution modes
- The local distributor runs tests serially on the local machine, and the HTTP distributor sends work to local or remote workers that can run tests in parallel.checked 4 Oct 2026
- Extensibility
- Users can extend Cosmic Ray with their own mutation operators exposed through plugins.checked 4 Oct 2026
- Install
- The tutorial says to install Cosmic Ray with `pip install cosmic-ray`.checked 4 Oct 2026
- Intended users
- The documentation presents Cosmic Ray for Python projects whose teams want to check whether their test suites meaningfully verify code behavior.checked 3 Oct 2026
- License
- The project repository identifies the software as MIT licensed.checked 4 Oct 2026
- Mutation filters
- Included filters can skip mutations by operator name, a `# pragma: no mutate` comment, or whether the mutated line changed relative to a Git branch.checked 4 Oct 2026
- Notable limitation
- Cosmic Ray may mutate test code if it shares a module with the production code under test; the documentation recommends keeping tests separate or excluding their modules.checked 4 Oct 2026
- Parallel execution
- The HTTP distributor sends mutation and test requests to workers that can run locally or remotely; each worker needs its own copy of the code under test.checked 3 Oct 2026
- Purpose
- Cosmic Ray is a mutation testing tool for Python 3 that makes small changes to production code and runs the test suite against each change.checked 4 Oct 2026
- Reports
- Cosmic Ray provides `cr-report` for session summaries and `cr-html` for generating an HTML report that shows the mutation used.checked 4 Oct 2026
- Runtime consideration
- Mutation testing can take a long time because the tool runs the test suite for each mutation; the documentation gives an example where 1,000 mutations and a 10-second test suite take about 2.7 hours.checked 3 Oct 2026
- Runtime limit
- A configured timeout ends a mutant test that runs too long, and Cosmic Ray marks that mutant incompetent.checked 4 Oct 2026
- Target users
- The documentation describes Cosmic Ray as a tool for Python projects and says it has been used on projects ranging from assemblers to oil exploration software.checked 4 Oct 2026
- Test commands
- A TOML configuration specifies the modules to mutate, test command, timeout, exclusions, and distributor.checked 3 Oct 2026
- Testing value
- Mutation testing checks whether tests detect changes in behavior, while code coverage indicates whether a line was executed.checked 4 Oct 2026
- Test runners
- The documentation shows test commands using `unittest` and says `pytest` and `nose` can stop after a failing test with the `-x` option.checked 4 Oct 2026
The company
- Maker
- The project documentation credits Sixty North AS.checked 4 Oct 2026
Alternatives to Cosmic Ray
- 1
Mutatest
Free planOpen sourceAPILinuxmacOSSelf-hosted7.2Free plan - 2
ArcMutate
Free planLinuxmacOSSelf-hostedWindows6.7From $8 a month - 3
Mutant
Free planLinuxmacOS6.7From $250 a year - 4
MuJava
Free planLinuxmacOSWindows6.3Free plan - 5
pymute
Free planLinux6.3Free plan - 6
go-mutesting
Free planLinuxSelf-hosted6.2Free plan