We ran npx supercov security on 72 repositories with known, labelled vulnerabilities and scored it next to ten other scanners on the same code.

In short: Supercov finds 56% of the labelled vulnerabilities in a median of 10 seconds, for about 5 cents a repository. The strongest AI agents find about 80%, but take 5 to 9 minutes and cost about $4.

Results

72 repositories from RealVuln, 2,016 labelled vulnerabilities, scored with the benchmark’s own rule.

Scanner	F1	Precision	Recall	Median time	Cost per repository
Codex CLI, GPT Daybreak Blue	0.71	63%	82%	5.4 min	$3.78
Codex CLI, GPT-5.6 Sol	0.70	62%	79%	8.8 min	$3.87
Kolega DevSec Max 0.1.0	0.69	56%	91%	not recorded	not recorded
DeepSeek V4.1 Flash agent	0.51	53%	50%	1.6 min	not recorded
GLM-5.3 agent	0.51	48%	54%	7.9 min	74¢
Claude Code, Sonnet 5	0.49	57%	44%	2.8 min	95¢
Codex CLI, GPT-6 Astra	0.49	45%	54%	6.4 min	$6.99
DeepSeek V4 Flash agent	0.48	59%	41%	2.6 min	2¢
DeepSeek V4 Pro agent	0.46	63%	37%	3.8 min	7¢
Supercov 2.0.1	0.41	33%	56%	10 s	5¢
Semgrep, auto rules	0.11	14%	9%	8 s	free

Recall is the share of labelled vulnerabilities a scanner found.
Precision is the share of its findings that were real.
F1 combines the two into one number.

Supercov is the fastest AI-based scanner by a wide margin, and one of the cheapest. Only Semgrep, which matches rules instead, is faster. It finds more than every scanner here except the two strongest Codex CLI agents and Kolega. A third of its findings are real, so its F1 is the lowest of the AI-based scanners.

What it finds, and what it misses

Vulnerability	Found
Open redirect, server-side request forgery	94%
Unsafe code execution	88%
SQL, command and other injection	86%
Path traversal	82%
Hard-coded secrets	79%
Mass assignment	72%
Cross-site scripting	68%
Weak authentication	57%
Insecure configuration	55%
Sensitive data exposure	54%
Weak cryptography	52%
Missing authorization	31%

It is best at problems visible on one line: a query built from user input, a path taken from a request. Missing authorization is its weakest area: the check that should protect a route usually lives in another file, so Supercov often flags the whole file rather than a line. 269 of the labels are kinds Supercov does not check for at all, such as missing rate limits.

It also reports the most false positives on the benchmark’s decoy findings: 26, against 1 to 23 for the others.

By language

Language	Repositories	Supercov F1	Best F1
Python	32	0.54	0.82
TypeScript	27	0.36	0.65
JavaScript	13	0.34	0.74

Python is its strongest language. On JavaScript, many of its false positives are in bundled third-party libraries, such as copies of jQuery or Bootstrap, that it should skip.

When to use something else

You want the most findings and can wait. A Codex CLI agent or Kolega finds 80 to 90%. Run it before a release, not on every change.
You want low cost and can wait a few minutes. A DeepSeek V4 Flash agent is cheaper and more accurate, at about 2.6 minutes a repository.
You want fast feedback on every change. Supercov answers in seconds, so your agent can run it after each edit with npx supercov security patch.

The tools also combine well: Supercov on every change, an agent-based scan before a release.

How we measured

Benchmark: 72 of RealVuln’s 140 repositories, the half we did not tune the checks on, each at the commit its labels were written for. They are deliberately vulnerable Python, TypeScript and JavaScript apps, so results on production code will differ.
Supercov: version 2.0.1 with its default settings, one repository at a time on a laptop. Cost is Jev’s list price of $42 per billion input tokens.
Other scanners: their findings, times and costs are RealVuln’s own published runs, and their times come from different machines. We timed Semgrep ourselves, with the same version and settings, on the laptop that ran Supercov. GLM-5.3 had 3 failed runs, counted as finding nothing.
Scoring: a finding counts when it names the right file, a matching vulnerability type and a line within 10 of the label. A finding for a whole file, without a line, counts once for a labelled vulnerability of its type in that file, as RealVuln’s own matcher allows.

Download the results or everything needed to rerun it: Supercov’s report for every repository, the times and the scoring script. Run September 25, 2026; rescored October 1, 2026 to count findings without a line.