Mutatest

Mutatest is a free command-line mutation-testing tool for Python projects. It is best for developers who want to investigate whether tests catch small code changes and can use pytest. Coverage filtering and configurable runs make it useful for focused diagnostics, though repeated test runs limit its fit for frequent, whole-codebase checks.

Overview

Mutation testing probes a test suite by introducing small changes and checking whether tests detect them. Mutatest generates valid mutants from Python’s Abstract Syntax Tree grammar and applies them to local __pycache__ files instead of source files. For developers who want to assess test effectiveness without changing source files, that is a practical distinction.

When coverage data is available, runs can target covered lines, concentrating effort on code the suite reaches. Mutation locations and candidate changes are sampled randomly, but a seed makes a run reproducible. Since the process can require repeated test-suite runs, Mutatest is better suited to deliberate diagnostics than an exhaustive check on every commit.

Key features

Configurable mutation checks

The command line supports run modes, mutation-category selection, file exclusions, timeouts, reports, and thresholds for surviving mutants. Operators cover augmented assignment, binary operations, bitwise comparisons and shifts, Boolean operations, comparisons, conditionals, indexing, names, and slices. This range and an API for custom controls give users room to tailor an investigation; users seeking a simple pass over a large codebase may find the repeated runs a poor trade-off.

Coverage and automated runs

Coverage data can narrow mutation targets to lines marked as covered. A configured survivor threshold can raise an exception when enough mutants survive, making the result usable as a quality gate in an automated stage. That offers a way to flag weak tests, but the documented warning about runtime makes broad per-commit checks less suitable.

Runner and Python constraints

The documentation describes supplying test commands, but only pytest has been tested. Installation requires Python 3.7 or 3.8; multiprocessing is supported on 3.8, while parallelization is unavailable on 3.7. Those constraints make Mutatest a more natural fit for teams already using the documented Python versions and pytest than for projects built around another runner or newer Python releases.

Pricing

Mutatest costs nothing. It is free and open source under the MIT License, with mutation checks, survivor reports, and quality-gate support included. There is no paid tier or trial to weigh against a limited free plan, so it suits individual developers and teams that can work within its platform and runner constraints.

Platforms

The project reports testing on Linux, Windows, and macOS with Azure pipelines. Python 3.7 and 3.8 are the documented installation requirements, with parallel execution limited to Python 3.8.

Who it's for

Mutatest fits Python developers who want to find gaps in a pytest suite, focus on covered lines, and use survivor thresholds in a diagnostic or CI stage. It is less suitable for teams that need a tested runner other than pytest, depend on parallel execution with Python 3.7, or expect to run mutation checks over the entire codebase on every commit.

Pros and cons

• Focused targeting: Coverage filtering can direct mutation checks toward executed lines rather than spending effort on uncovered code.
• Reproducible sampling: A seed can make random trials repeatable, helping teams revisit a diagnostic result.
• Useful automation controls: Survivor thresholds can raise exceptions, and reports and timeouts help shape automated runs.
• Narrow tested runner support: Only pytest has been tested, which limits confidence for projects using other test commands.
• Potentially slow at scale: Repeated suite runs make exhaustive, frequent checks a poor fit.
• Version-dependent parallelism: Multiprocessing is supported on Python 3.8 but not Python 3.7.

Alternatives

For a free Python-focused option that requires Python 3.11 or newer and runs on Linux, consider pytest-gremlins. If open-source mutation testing for JavaScript or TypeScript, C#, or Scala is the priority, Stryker Mutator is the more relevant choice.

ArcMutate offers a free open-source-project plan and a free trial, alongside paid pricing; it may suit readers who want to compare a freemium alternative. Mutant has a free full-featured plan for public repositories and a commercial monthly plan billed per developer, making it an option for readers weighing public-repository use against paid commercial use.

pymute is another free option for Linux users willing to install it with Rust and Cargo. Infection, Gambit, and mutmut are also free alternatives. Browse the Mutation Testing Tools category for more options.

Verdict

Choose Mutatest if you need a free, configurable way to diagnose whether Python tests catch small changes, particularly when coverage targeting and survivor thresholds matter. Its main draw is the control it gives focused mutation runs; look elsewhere if you need a tested runner beyond pytest or fast, exhaustive checks on every commit.