Mutation testing

Find the weaknesses in your passing tests.

A passing suite tells you what its assertions checked. Mutation testing asks a useful follow-up: would those tests notice a change in behaviour? With Cue April, run approved Stryker mutation checks alongside your other tests and see which deliberate code changes survive.

What is mutation testing?

Mutation testing checks whether your tests notice small, deliberate edits to code, called mutants. It runs the tests against each edit. A mutant that survives may expose a missing assertion, an unclear requirement or a change that makes no relevant behavioural difference.

A passing suite can miss an important boundary.

The fictional requirement is free delivery from 5000p. The correct function uses >=, but these two tests check only 4999p and 5001p. Both pass when that operator is changed to >. Stryker makes that controlled source change so we can see whether the suite notices it.

src/delivery.mjs: the correct code under mutation test
// Fictional requirement: delivery costs 500p below £50 and is free from £50.
// The example accepts a non-negative integer basket value in pence.
export function deliveryPence(basketPence) {
  return basketPence >= 5000 ? 0 : 500;
}
test/weak.test.mjs: the two-example suite
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { deliveryPence } from '../src/delivery.mjs';

test('charges below the threshold', () => assert.equal(deliveryPence(4999), 500));
test('is free above the threshold', () => assert.equal(deliveryPence(5001), 0));

Run Stryker on one source file.

The download pins Stryker 10.0.0 and uses its command runner with Node’s built-in test runner. This keeps the example small: one source file, two tests and a single mutation worker. Coverage analysis is disabled because the command runner executes the suite as a command. Stryker’s configuration reference explains these options.

stryker.weak.json
{
  "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
  "mutate": ["src/delivery.mjs"],
  "testRunner": "command",
  "commandRunner": { "command": "node --test test/weak.test.mjs" },
  "coverageAnalysis": "off",
  "concurrency": 1,
  "reporters": ["clear-text", "json"],
  "jsonReporter": { "fileName": "results/mutation-weak.json" },
  "tempDirName": ".stryker-tmp-weak"
}
Run from the downloaded example folder with Node 24
npm ci --ignore-scripts
npm run test:weak
npm run test:mutation:weak

Investigate the survivor before changing a test.

The native weak-suite report contains five mutants: four were killed and one survived. The survivor changes basketPence >= 5000 to basketPence > 5000. The independent requirement says exactly 5000p gets free delivery, so this survivor identifies a missing boundary assertion. Other survivors can be equivalent changes or uncover unclear requirements; their status alone does not decide what to change.

Observed local results from the two native JSON reports
Two-test suite:   5 mutants, 4 killed, 1 survived
Boundary mutant:  basketPence > 5000
Status:           Survived

Three-test suite: 5 mutants, 5 killed, 0 survived
Boundary mutant:  basketPence > 5000
Status:           Killed

Add the boundary assertion and run again.

The stronger suite adds the exact threshold. Its fresh Stryker run killed all five mutants, including the former survivor. That is a result for this tiny fixture, not a universal mutation-score target or a claim about customer code. Both native reports and source variants are in the download.

test/strong.test.mjs: preserve the requirement explicitly
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { deliveryPence } from '../src/delivery.mjs';

test('charges below the threshold', () => assert.equal(deliveryPence(4999), 500));
test('is free at the threshold', () => assert.equal(deliveryPence(5000), 0));
test('is free above the threshold', () => assert.equal(deliveryPence(5001), 0));
Run the stronger suite and compare reports
npm test
npm run test:mutation:strong
# Inspect results/mutation-weak.json and results/mutation-strong.json

Choose the next assertion from the requirement.

Mutation testing works best on a bounded rule whose expected behaviour is clear. Review each survivor against that rule, then decide whether to add an assertion, clarify the requirement or document an equivalent mutation. Keep errors, timeouts and untested scope visible. The recorded sample report is a separate owned fixture with twelve mutations and six survivors; its totals should not be combined with this guide’s five-mutant example.

Use mutation evidence in a release review.

Cue April supports approved Stryker checks alongside other testing methods and retains the native outcomes for review. Agree the source scope, runner setup and compute allowance first. Property-based checks investigate input combinations, while mutations challenge the assertions that should catch changed behaviour. Your team reviews the findings and decides what ships. Review ways to start.

Primary documentation

Framework behaviour and configuration are documented by their maintainers.

Questions about this testing method

01

Does every surviving mutant mean there is a bug?

A survivor needs investigation. It may expose a missing assertion, an unclear requirement or a change that makes no relevant behavioural difference. The report keeps that uncertainty visible.

02

Can I inspect a real sample?

The public owned sample retains twelve generated mutants, including six survivors. You can inspect its measured outcomes and evidence pack before discussing an evaluation.

03

What is a good mutation score?

There is no universal target. A mutation score describes only the code and mutants that ran, so compare it over time on the same scope instead of across projects. Treat each surviving mutant as a prompt to check an assertion or a requirement.

04

Is mutation testing worth it?

It pays off most on code where a missed bug is expensive, such as pricing rules or access decisions. Mutation runs take longer than a normal test run, so start with a focused slice of the repository instead of the whole codebase.

05

What are the disadvantages of mutation testing?

It costs more compute than a normal run, because the tests run against every mutant. Some mutants make no behavioural difference and survive for a good reason, so each survivor needs review before anyone changes the tests.

Your next release starts here

Keep building.
Put testing to work.

Give Cue April your next testing challenge. Let your AI quality lead coordinate the checks, so your team can focus on what comes next.

Create account

Create your account, connect a repository and review your proposed setup before approving a run.