Testing AI-generated code

Check AI-generated code against your requirements.

When an AI coding tool produces a change and its tests, review what those tests actually establish. Give the change an approved behavioural expectation, then investigate it with complementary checks and evidence your team can inspect.

How do you test AI-generated code?

Testing AI-generated code means checking the change against requirements your team agreed independently, so the code and its tests do not simply repeat the same mistaken assumption. Several methods then check the change, and mutation testing shows whether the tests notice deliberately introduced faults.

Separate the requirement from the implementation.

Suppose a ticket says delivery is free from £50. A function using > and tests covering £49.99 and £50.01 can agree with each other while missing exactly £50. This is a deliberately constructed teaching example, rather than a recorded evaluation of a coding assistant. It demonstrates the review problem regardless of who wrote the change.

The deliberately seeded implementation defect
// Deliberately seeded boundary defect: exactly £50 incorrectly costs 500p.
export function deliveryPence(basketPence) {
  return basketPence > 5000 ? 0 : 500;
}
The missing boundary assertion
assert.equal(deliveryPence(5000), 0);

Use checks that can disagree with the original tests.

Start from the requirement, then ask a different question. The generated property says every basket from 5000p to 10000p should get free delivery. Its local fast-check run found 5000p at seed 20261015 and path 26. The Cucumber examples also failed at that boundary. Against the correct code, Stryker’s >= to > mutant survived the two-example suite and was killed after the boundary assertion was added.

  • The property found the concrete failing input in its sampled domain.
  • The scenario exposed that failure as an agreed business example.
  • Mutation testing showed why the original two-example suite could miss it.
  • These methods describe the same boundary fault; they are not three independent product defects.
Run the complete verified educational example with Node 24
npm ci --ignore-scripts
npm run verify

Keep the recorded output with the change.

The verification harness expects the deliberately buggy property and scenario commands to fail, checks the counterexample and compares both native mutation reports. It exits successfully only when those expected outcomes are observed. The corrected property passed 1,000 generated cases; the strong suite killed five out of five mutants in this fixture. Inspect all source and recorded results. For separate evidence from Cue April’s actual engines, inspect the owned local product run, which records unit, API and generated-input outcomes against a tested product revision.

Observed local outcomes, summarised
fast-check, buggy: counterexample [5000], seed 20261015, path "26"
fast-check, fixed: 1000 generated cases passed
Cucumber, buggy:   2 scenarios passed, 1 failed
Cucumber, fixed:   3 scenarios passed
Stryker, weak:     4 mutants killed, 1 survived
Stryker, strong:   5 mutants killed, 0 survived

Review this checklist before merging.

A useful review checks the change and its evidence against the ticket. GitHub’s responsible-use documentation likewise says generated tests need review. Keep human judgement on the expected behaviour and the implications of a failure.

  • Write the intended rule and important exceptions before evaluating the generated assertions.
  • Review the diff for permission changes, validation, external effects and data handling.
  • Test exact boundaries and denied actions, not only representative successful cases.
  • Check that an assertion would fail if the relevant behaviour broke.
  • Reproduce a reported failure and keep its input, requirement and tested revision.
  • Keep errors, missing cases and blocked checks separate from passes.
  • Rerun relevant checks after the fix or a new commit; follow the repository’s merge rules.

Choose the next method for the risk in the change.

Use property-based testing for a stated rule across inputs, mutation testing for the strength of its assertions and Gherkin scenarios for agreed examples. Add API or browser checks when the risk crosses those interfaces. The input domain, environment and expected result should be explicit; no finite set of checks guarantees that every defect has been found.

Bring your own code into a scoped evaluation.

Cue April can coordinate approved checks and retain findings, revision identity and incomplete work for your team to review. Account setup and repository connection are separate from approving execution. Supported tools, qualified runners and paid terms are confirmed for your scope. Compare self-serve setup and managed testing, or see how evidence fits a pull request.

Primary documentation

Framework behaviour and configuration are documented by their maintainers.

Questions about this guide

01

Does this require a particular coding assistant?

The testing workflow evaluates the approved repository change and supported runtime. Compatibility is assessed through the source, test configuration and execution environment rather than the brand of the coding assistant.

02

What should I inspect before merging?

Review the exact tested revision, required method outcomes, reproduced findings and incomplete obligations. Confirm that the evidence still applies to the current change and follow your repository’s review rules.

03

How do you evaluate AI-generated code?

Start from requirements your team agreed independently, then check the change with several methods: existing unit tests, API or browser checks, and mutation testing to see whether the tests would notice a fault. Review generated tests as carefully as generated code.

Your next release starts here

Keep building.
Put testing to work.

Give Cue April your next testing challenge. Let your AI quality lead coordinate the checks, so your team can focus on what comes next.

Create account

Create your account, connect a repository and review your proposed setup before approving a run.