SupercovDocumentation

Changelog

What changed
in Supercov

Every published release, newest first. Each entry is the release note written when that version shipped.

Latest 4.22.0 · npx supercov@latest --version

4.22.0

· Release notes

Added

  • Native bun:test coverage attribution for individual test bodies, including async and concurrent tests, parameterized rows, and done callbacks. Lifecycle hooks are recorded separately as setup. Existing Bun options and configured preloads are preserved. Callback outcomes and native Bun verdict limitations are described in supported suites.

Fixed

  • Instrumented application modules initialize the runtime before their first probe under bun test, including projects with browser configs, nested frontends, aliases and preloads. TypeScript browser sources also receive the runtime fallback without requiring a Dockerfile. Native bun test no longer selects an unrelated browser runner from an unused package script (#176).
  • Coverage without test attribution no longer implies that a process ran in a container or VM. Interrupted Bun callbacks retain their named coverage and assertion-phase metadata with an unknown outcome (#177).
  • Concurrent Docker container creation no longer interleaves diagnostic journal records and loses warnings, including the warning about an image built from an earlier source copy.

Contributors

4.21.0

· Release notes

Added

  • A run whose tests failed loading files the project has and its isolated copy does not, such as dist/ an earlier build made, names those files and says why: the copy leaves out build output, and nothing built it there. The failures read as broken tests before, with build=0ms in the timings the only sign.

Changed

  • A Rust for loop over a non-empty array literal, a constant of the file that is one, a range between integer literals or constants of the file that are one, a split, iter::once, or a chain with one of these on either side (for x in [a, b], for k in KINDS, for i in 0..8, for copy in 1..=MOST, for part in text.split(','), for phase in steps.iter().chain(once(&last)), through iter, enumerate, map and the like) counts only its entry, not a zero-iterations outcome it can never take, as a JavaScript loop over an array literal already does.
  • A Rust run of other tests than the run before it (cargo test -- other_test) reuses that run’s instrumented build; anything after the command’s -- used to make Supercov instrument and build the project again.

Fixed

  • A browser suite with no JavaScript source whose tests drove servers in two languages, such as Puma beside a Python server, published a run per server that named neither of the others: runs <id> said nothing of the run beside it. Each names the other now, as a browser run and its servers’ runs already did.
  • supercov -- cargo test ran the tests without a target runner Cargo is configured with when Supercov could not read the runner, such as one that leads to the crate itself, and they passed where plain cargo test fails. The run is refused, naming the configuration, as cargo nextest run already was.
  • A Rust run ignored SUPERCOV_COMMAND_TIMEOUT_MS: a hung test kept it running for good, and a value of it or of SUPERCOV_DIAGNOSTIC_INTERVAL_MS that is no number of milliseconds was not refused. It now refuses such a value before starting anything and keeps the deadline as other languages’ runs do: past it, the tests still running are stopped, no other is started, what ran is published (nothing, where no test reported), and the run exits 124.
  • An assertion question about the last value of a Kotlin if or when branch that ends a lambda whose value is used (val x = run { if (c) { log(); 4 } else 5 }) asked what happens if the value is skipped, a change that would not compile; it asks what happens if the branch yields the default value (null, 0 or false) instead, as for any other branch value. A lambda whose value nobody takes (run { .. } on its own) is asked as before.
  • A Cargo runner, compiler or wrapper configured by a relative path out of the workspace (runner = "../tools/run") was refused on Linux and macOS and accepted on Windows. It is run from where the path leads on every platform, as Cargo runs it; a path that leads back into the workspace is read from the copy, and one that is the workspace itself is still refused.
  • runs <run> assertions panicked when the run’s saved assessment was valid JSON but no object, as a file edited by hand or replaced can be; such a file is read as no assessment, as one that is not JSON already was.
  • Supercov panicked when its output stopped being read part way, as supercov -- npm test 2>&1 | head or supercov runs latest gaps | head stops reading it; it goes on without printing what is no longer read.
  • A JavaScript or TypeScript file Supercov could not parse was reported one column to the right of the error.
  • security read a line question the provider refused as finding nothing on those lines; their lines are counted as not checked one by one, as lines whose question did not fit already were.
  • A --json page cut to fit one reply could still be refused as too large: the note saying where the rest starts was added after the page was fitted. The page is fitted with its note.
  • A container time from the Docker daemon with anything but digits after its seconds made the Docker relay panic while reading it; it is read as no time.
  • A test command that could not start says whether there is no such program or it may not be run, before the system’s own words, and exits 127 or 126 by that on every platform rather than by the message’s wording; a step of Supercov’s own that the system refused no longer reads as a file that may not be run.
  • A Kotlin file with a class body on one line (class C { val x = 1 }), two members on one line, or a line starting with an identifier that begins with a keyword (set_value = ...) was read as not parsing and left unmeasured; Supercov now builds the Kotlin grammar itself with upstream’s unreleased fixes for these. A file ending inside an annotation could stop the run for good; it now ends.
  • Kotlin files using context parameters, when guards, multi-dollar strings, suspend { }, fun interface, actual constructor, annotated accessors, and two dozen other constructs were read as not parsing and left unmeasured. Supercov now keeps its own copy of the grammar and checks it against every Kotlin file of six real projects (Exposed, kotlinx.coroutines, ktor, moshi, Now in Android, OkHttp): 165 of their 5,429 files were left unmeasured, now 3, two of them templates that are not Kotlin.
  • assertions assess and its --dry-run estimate took minutes before printing anything on a run whose tests share most of what they run, as a merged or end-to-end run’s do: each request and each cache key worked out its test’s code and its files’ functions again. They are worked out once, and the requests are the same: 400 tests sharing 6,000 statements went from 166 s to 4 s on one core.
  • A measured wrangler dev whose wrapped configuration could not be written left the files it had written beside the configuration; they are removed, and the configuration is used as it is.
  • runs <run> assertions assess --changed said nothing of requests Jev did not answer: with every request failing it printed the request count and exited 0, leaving tests affected nothing to go on. It now ends with why when none was answered, and says how many failed, their statements not asked, when some were.
  • A Rust expression statement behind a #[cfg] the target leaves out (#[cfg(windows)] continue; on Linux) was counted as a line no test runs; it is left out, as a let or an item behind one already was.
  • merge left out a run whose test files had changed and whose record of its files could not be read without saying so; it names the run and why, as for every other run it leaves out.
  • SUPERCOV_SOURCE_ROOTS naming several roots that do not exist said “a, b does not exist”; it says “do not exist”.
  • SUPERCOV_SOURCE_ROOTS naming a directory outside the project was refused as “not a regular file or directory”; it is refused as outside the project.
  • merge refused a run that holds no record of its files, where the latest run holds one and nothing else differs, with an empty list (“differing domains: “); it names the record (“differing domains: recorded files”).
  • import said a file of the project that is not UTF-8 text was “no such file in the project”; it says the file is not a text file.
  • A browser suite whose SUPERCOV_SOURCE_ROOTS named only its JavaScript was refused when the project also held Go, Java or Kotlin source outside those roots (SUPERCOV_SOURCE_ROOTS=src left no Go source to measure). Such a server is left unmeasured, as a Python or Ruby one already was.
  • Supercov panicked as it started when its own path was not UTF-8 text, as a directory name on Linux can be, and when an argument was not. It starts, and refuses such an argument by name; a cargo nextest run that has to name Supercov as its runner says the path is not UTF-8.

4.20.3

· Release notes

Fixed

  • runs <run> files and gaps listed a file the run left out, such as a Worker no test ran or a module no build compiled, as covered; they say it is not measured, and why.
  • scope and the run’s summary called a run complete whose Worker no test ran; that Worker leaves the run’s measurement incomplete, as runs check already judged it.

4.20.2

· Release notes

Changed

  • A Rust Worker no test of a run started, or whose tests loaded no build of its measured source, is not measured, instead of counting every line as one the tests missed: its files leave the run’s totals, the file view gives the reason at the top, the run names them after the tests’ output, and runs check refuses to judge a floor over such a run.
  • A build made before the run is named in that reason: Supercov builds nothing itself, so a Worker the tests load from an earlier build/ is built in the measured command or the tests.

4.20.1

· Release notes

Changed

  • A Rust Worker that cargo test starts is measured whatever configuration the test gives wrangler dev, wherever it is and whatever its main imports: a test kit that writes its own configuration per test, with a module of its own that loads the built Worker, is measured as the crate’s own configuration is. Supercov hands wrangler dev a copy of the configuration beside it, removed when wrangler exits, and leaves every configuration as it was written.
  • SUPERCOV_RUN_ID is set in every process a measured Rust, Go or Java/Kotlin command starts, as it already was for JavaScript, Python and Ruby, and is documented as the way a test knows it is measured.

Fixed

  • A test that copied a Worker crate’s wrangler.toml as a template for its own got a main naming Supercov’s entry, which wrangler could not find beside the copy; the crate’s configuration is no longer rewritten.
  • Where tests started wrangler dev and no measured build of a Worker crate ran in it, its files name each wrangler dev -c <path> the tests started and why one was not measured, instead of saying no test started a Worker.
  • wrangler’s .wrangler/ state in the copy was synced back into the project; it stays in the copy.
  • cargo test -- --nocapture under Supercov showed nothing a test printed; each test runs without capture and its output is shown as it is written.

4.20.0

· Release notes

Added

  • Cloudflare Workers in Rust go further: code in a Durable Object is measured and credited to the test whose requests reached it, and every Worker of one wrangler dev -c a/wrangler.toml -c b/wrangler.toml session is measured, read through workerd’s own inspector. A Worker crate is measured as worker-build compiles it (wasm, a release unless --dev, with its features), counts whatever cargo test compiles, and runs that started different Workers merge.
  • JavaScript and TypeScript code under no source root that first-party source imports, such as a shared/ folder beside src/, counts as source with no SUPERCOV_SOURCE_ROOTS needed.
  • Kotlin for loops and Go loops that name only a condition record their branch and MC/DC outcomes.

Changed

  • A bug in Supercov’s JavaScript or Rust instrumenter on one file no longer fails the run: that file runs as written, and the run says it was not measured and why.
  • Queries no longer re-hash evidence that has not changed since it was verified (0.07 s instead of 0.30 s on 150 MB), and reading a run’s record of its source no longer loads the whole archive.
  • A command ended by a signal exits 128 plus the signal in every language, one that cannot start exits 127 or 126, and one that fails before any test ran keeps its own status.
  • minimize without --target keeps everything the tests together cover.

Removed

  • The old Rust frontend’s scope fields (unit, measurementComplete), leftovers of the assertion-map confidence, Python evidence transport version 1, and cache locations only old releases wrote.

Fixed

  • Ruby files loaded after Bootsnap 1.26.1’s setup ran without Supercov’s probes, so a Rails app counted no branches; Supercov’s loader stays in front. Code called inside a Ractor, and a local first assigned with ||=, no longer fail under measurement.
  • Java and Kotlin conditions that open with a comment, instanceof pattern conditions, TestNG tests beside the JUnit Platform, and Gradle builds whose root holds only settings.gradle are measured and compile.
  • make test and other wrappers around Go, Gradle or Maven no longer receive flags the wrapper refuses.
  • Rust: a Cargo configuration’s build.rustflags apply again when RUSTFLAGS is unset, cargo +<toolchain> test builds doctests with that toolchain, a runner named with a backslash is found on PATH, an attributed boolean let is instrumented, and a workspace’s exclude and one-line members are read correctly.
  • tests affected follows package.json subpath imports (#lib/x), imports of a package’s own directory and aliases from an extended tsconfig, and a runner configuration added since the run reaches only that runner’s tests.
  • security reads files near the request size limit in windows, names the paths and callers it did not ask about, and no longer reports paths through default imports or aliases that do not reach the function; quality and security refuse a --run that names no run before asking anything.
  • Damaged indexes, evidence archives, region files and browser driver replies claiming enormous sizes are refused instead of aborting, allocating or running for a long time.
  • Projects holding a symbolic link loop can be measured.
  • Many query answers are clearer: pages past the end say so, counts of one are singular, stale warnings name only the files read, diff, kinds, runners and the summary honour --kind and --runner, unknown words and flags are refused by name, usage mistakes under --json answer in JSON, and file and line give the scope’s reason for a file the run left out.

4.19.0

· Release notes

Added

  • wasm-pack test is measured: --node, and --headless --chrome, where Supercov reads each test’s page through the driver before the runner closes it. Each wasm test is credited its own lines and decisions. Tests run in Firefox, Safari or Edge are run and reported as not measured.
  • cargo nextest run measures a Rust Worker its tests start with wrangler dev, as cargo test does.

Changed

  • A Rust Worker crate no test ran says why in gaps and the file view: no test started it with wrangler dev, or a test started it with wrangler dev --remote, where it runs on Cloudflare’s edge and cannot be read.

Fixed

  • Files a project’s .gitignore ignores, such as .zig-cache/ build output, no longer make Rust runs stale.
  • A container a test created through Supercov’s Docker relay could lose its note (for example that its image was built from an earlier copy) when the client finished quickly.

4.18.0

· Release notes

Added

  • Rust Workers are measured. A Playwright suite that starts one with wrangler dev credits each test the Worker work done while it ran. A cargo test test that starts wrangler dev --local itself is credited exactly what its Worker ran, and the Worker crate counts though cargo test never compiles it.
  • Rust compiled to wasm is measured: cargo test --target wasm32-unknown-unknown under wasm-bindgen-test-runner, --target wasm32-wasip1 under wasmtime or wasmer, and wasm-bindgen crates that node:test, Vitest, Jest or a Playwright page load, each test credited its own lines and decisions.
  • Containers started by any Docker client are measured, not only Compose’s: docker run, Testcontainers (Ryuk included) and Docker SDKs go through a relay Supercov puts on DOCKER_HOST.
  • supercov import <tracefile> --test <name> adds line coverage recorded where Supercov cannot run (an LCOV file) to a run, as a new run. It is credited by line only, to a test of its own or to the background. An LLVM profile is exported as LCOV with LLVM’s tools first.

Changed

  • Rust code a #[cfg] leaves out of every build (a feature that is off, cfg(loom), a build script’s --cfg) is outside the denominator instead of counted uncovered. Rust probes record into one mapped region per process, and covered counts are unchanged.
  • Assessment answers survive dependency and configuration changes. A changed lockfile, package.json or test configuration asked the whole suite again; now only questions whose text changed are asked. Answers saved by earlier releases are asked once more.
  • A multi-location line query answers every location it can and lists the rest under “Not answered” instead of failing.
  • line, file, decision and source warn only about files they read that changed since the run, and name them. A page cut to fit one reply says where the rest start. patch on a failed suite still shows the change’s coverage.

Fixed

  • A JavaScript program that hands itself over with process.execve, or a Ruby one with exec, lost everything it had run; it is kept now.
  • cargo test failed in a workspace with a cdylib-only or binary-only crate.
  • Supercov returned the instrumented Worker that wrangler dev built in its copy to the project’s build/, where a build that embeds it would ship it. It stays in the copy.
  • wrangler’s .wrangler/ state and LLVM profiles (default.profraw) in the project no longer make Rust runs stale.
  • The file view of a file the run never measured said “every line is covered and measured”; it now says nothing was measured.
  • A filtered assertions reading counted statements none of the picked tests was asked about as not asserted, and showed 0% for test files whose tests assert. They are now counted apart.
  • The judge misread Jest and Vitest null checks: toBeNull() fails on undefined, so a return null changed to undefined is caught.

4.17.2

· Release notes

Fixed

  • Supercov could change what a Python program computed. A value made of three or more or/and operands (path = given or os.environ.get("X") or shutil.which("x")) came back as True instead of the operand, in every measured file after the first one with decisions, so code under test took other paths and tests failed that pass without Supercov.
  • A Jest project whose jest-config package sits only under jest-cli or @jest/core (npm’s layout beside next/jest) ran under Jest’s defaults: its configuration was not read, so tests it leaves out (testPathIgnorePatterns) ran too. It is now read through the Jest that runs.
  • A Jest test that opens with a docblock (/** @jest-environment jsdom */) ran in Jest’s default environment (“window is not defined”): Supercov’s loader went in front of the comment. It now goes after it, on the same line.
  • A measured run left generated scripts in .supercov/workspaces/, so a project whose tests lint . (eslint .) failed after it. The copy now keeps only its caches.
  • runs clean during a run that had just started said “could not acquire the Supercov project lock”; it now says a run is active.
  • --limit 0 lists every entry in runs <run> queries (file, gaps, files, line, test and the rest), as it already did for assertions, quality and security.

4.17.1

· Release notes

Fixed

  • Assessment asked only one test about a statement at a module’s top level (const limit = 10;, const router = express.Router()): it runs once, when the first test loads the module, and was recorded for that test alone, often one an auth check turned away, so it read “1 of 1 asked, not asserted”. Every test that runs the module’s functions now counts as running it too, after the test that loaded it. JavaScript, TypeScript and Python.

4.17.0

· Release notes

Added

  • assertions and assertions assess take --kind, --runner, --tag, --title and --test-file, as every query does. assess --tag @e2e asks only those tests about the statements they ran (--dry-run estimates just that) and saves the result apart from the run’s whole assessment; assertions --tag @e2e reads what those tests assert.
  • tests affected takes the same filters: from a run that merged two runners’ suites, tests affected --names --runner playwright lists only Playwright’s affected tests, for its --grep.

Fixed

  • A Next.js project whose next.config sets turbopack.root to its own directory could not run Turbopack under Supercov (Next.js package not found, or a node_modules link “out of the filesystem root”): in the copy that root excluded the dependencies it links to. The root is now widened to hold them.
  • A node:test suite that mocks modules (--experimental-test-module-mocks) and maps import paths with a preload that patches Node’s resolver (--import tsx 4.20, tsconfig @/ paths) failed under Supercov with Cannot find module '@/...': Supercov loaded node:test before the command’s own preload, and Node’s mock loader kept the resolver it found then. The adapter now loads when a test first asks for node:test.
  • assertions assess refused a checkout that lacked any file the run had read, such as a git-ignored .env only CI generates. It now checks only the files its questions quote, each assessed statement’s source and each test’s own file, and still names any of those that changed.

4.16.0

· Release notes

Fixed

  • Django projects are recognised: supercov -- python manage.py test measures Django’s runner like unittest, each test credited with its own coverage, in projects that declare themselves only by manage.py and requirements.txt.
  • A Rust test run through a package script keeps its arguments: supercov -- npm test -- adds with "test": "cargo test" ran every test. The command is also split the way sh splits it, keeping empty arguments and backslashes in double quotes and joining line continuations.
  • Assessment shows each test its own code when tests in one file share a name: Python classes (TestCart::test_total, TestOrder::test_total), Minitest classes, JUnit @Nested classes, Rust inline modules and JavaScript describe groups.
  • A Rails-style Ruby test whose first word has four letters (test "adds numbers") is found for assessment; it was read as a numbered minitest/spec example.
  • A Next.js route handler at the root of app/ (app/route.ts) ties what it runs to the request that ran it, as deeper handlers do.
  • A Rust condition in an array length runs at compile time and is declared unmeasured, like a const fn body, instead of counting as a gap no test could close.
  • Rust assessment no longer asks about changes that cannot compile or change nothing: a raw pointer held in a local becomes std::ptr::null(), a closure with a return type keeps its braces, 0.0 becomes 1.0, None becomes Some(Default::default()), and a value spelled with its path counts as the same value.
  • A Python test deep in its file is sent with ... only where lines are left out, not between adjacent decorators.
  • A Java project with two build files is told once, not twice, that its build stopped failing on warnings.

Earlier releases

Notes for these are on GitHub.

Mirrored from GitHub releases. Older releases remain installable; see the install guide.