Assertion coverage — Supercov docs

A test can run a calculation without checking its answer. Assertion coverage finds those places. It works with JavaScript, TypeScript, Python, Ruby, Go, Rust, Java and Kotlin.

A passing test can miss a wrong result

This function confirms an order and calculates its total:

export function checkout(price, quantity) {
  const total = price * quantity;
  return { status: 'confirmed', total };
}

The test checks the status, but not the total:

import assert from 'node:assert/strict';
import test from 'node:test';
import { checkout } from './checkout.js';

test('confirms an order', () => {
  const order = checkout(25, 2);
  assert.equal(order.status, 'confirmed');
});

Every line runs and the test passes. It would also pass if the total were 49 instead of 50. The missing check is:

assert.equal(order.total, 50);

Now a wrong total fails the test. Line coverage hasn’t changed, but the test checks something it didn’t check before.

How Supercov finds it

Supercov records which statements each test ran. assertions assess then asks Jev, for each test and each statement it ran, whether a small change to that statement would make the test fail: the total computed differently, a condition inverted, a returned value dropped. A statement is asserted when at least one passing test that runs it would fail.

In the example, the status is asserted. The total is not: the test runs it, but nothing the test checks depends on it.

The answers are saved with the run and kept by the exact question, so after a commit only the questions the change touched are asked again. A first assessment costs about 0.007 cents a statement.

Try it in your project

Paste this into your usual coding agent:

Run our full test suite through npx supercov, then run
npx supercov runs latest assertions assess. Read
npx supercov runs latest assertions and pick one statement no test checks
where a wrong value would matter. Add an assertion for the expected behavior.
Only change tests; do not weaken existing checks. Rerun the same suite, assess
it again, and show the test change and the asserted share before and after.

The agent should show you the assertion it added and explain which wrong result would now fail. That explanation matters more than the score by itself.

Check assertion coverage yourself

Measure the tests, then assess the run. Assessing needs a TypeSafe AI API key in TYPESAFE_API_KEY; reading the result afterwards does not:

npx supercov -- npm test
npx supercov runs latest assertions assess
npx supercov runs latest assertions

For the checkout example:

Run run_35af70f11d81eaa6: 50% asserted (1 of 2 executed statements)

Files, most statements not asserted first: a statement is not asserted when none of the tests asked about it was judged to fail if it changed. Up to 18 of the tests that run a statement are asked, so a test never asked can still catch it.
 NOT ASSERTED  ASSERTED  FILE
            1       1/2  src/checkout.js
showing 1-1 of 1

Each listing ends with the next step down. One file lists what is not asserted in it, with the change Jev considered and the test that came closest:

npx supercov runs latest assertions src/checkout.js

Run run_35af70f11d81eaa6: src/checkout.js: 50% asserted (1 of 2 statements)

Not asserted: none of the tests asked was judged to fail if they changed. Up to 18 of the tests that run a statement are asked; one never asked can still catch it.
  LINE  STATEMENT  (change)  tests asked  closest test
     2  const total = price * quantity;  (value becomes undefined)  1 of 1  confirms an order 10%
showing 1-1 of 1

One statement shows every asked test’s answer:

npx supercov runs latest assertions src/checkout.js:2

Run run_35af70f11d81eaa6: src/checkout.js:2

  const total = price * quantity;
  Change: value becomes undefined
  Not asserted: 1 test(s) run it, 1 asked
     10%  would pass   confirms an order (test/checkout.test.js)

The percentage is how likely Jev judged the test to fail with the change made.

runs latest file and runs latest line show the same verdicts beside the coverage, and npx supercov report puts them in the HTML report.

Understand the score

Assertions is the share of executed statements that at least one passing test which runs them was judged to catch a change to. It is not a count of assertions, and statements no test ran are not part of it: line coverage already reports those.

A statement that is not asserted is a place to add a check, not always a missing test. The strength of a check matters: assert.ok(order.total) passes for both 49 and 50, so it catches no change to the total, while assert.equal(order.total, 50) does.

The verdicts are Jev’s judgment, not a run of changed code. Checked against running each change on four projects in Go, Rust, Java and Ruby, 85 to 96% of them were right. Supercov does not generate or execute mutated code.

A passing test can miss a wrong result

How Supercov finds it

Try it in your project

Check assertion coverage yourself

Understand the score

Read more in the terminal