We ran npx supercov security on 72 repositories with known, labelled vulnerabilities and scored it next to ten other scanners on the same code.

In short: Supercov finds 43% of the labelled vulnerabilities in a median of 10 seconds, for about 5 cents a repository. The strongest AI agents find about 80%, but take 5 to 9 minutes and cost about $4.

Results

72 repositories from RealVuln, 2,016 labelled vulnerabilities, scored with the benchmark’s own rule.

Scanner	F1	Precision	Recall	Median time	Cost per repository
Codex CLI, GPT Daybreak Blue	0.71	63%	82%	5.4 min	$3.78
Codex CLI, GPT-5.6 Sol	0.70	62%	79%	8.8 min	$3.87
Kolega DevSec Max 0.1.0	0.69	56%	91%	not recorded	not recorded
DeepSeek V4.1 Flash agent	0.51	53%	50%	1.6 min	not recorded
GLM-5.3 agent	0.51	48%	54%	7.9 min	74¢
Claude Code, Sonnet 5	0.49	57%	44%	2.8 min	95¢
Codex CLI, GPT-6 Astra	0.49	45%	54%	6.4 min	$6.99
DeepSeek V4 Flash agent	0.48	59%	41%	2.6 min	2¢
DeepSeek V4 Pro agent	0.46	63%	37%	3.8 min	7¢
Supercov 2.0.1	0.42	42%	43%	10 s	5¢
Semgrep, auto rules	0.11	14%	9%	8 s	free

Recall is the share of labelled vulnerabilities a scanner found.
Precision is the share of its findings that were real.
F1 combines the two into one number.

Supercov is the fastest AI-based scanner by a wide margin, and one of the cheapest. Only Semgrep, which matches rules instead, is faster. Supercov is also the least accurate of the AI-based scanners.

What it finds, and what it misses

Vulnerability	Found
Open redirect, server-side request forgery	89%
SQL, command and other injection	86%
Unsafe code execution	82%
Path traversal	79%
Mass assignment	64%
Cross-site scripting	56%
Hard-coded secrets	53%
Weak cryptography	48%
Sensitive data exposure	43%
Insecure configuration	43%
Weak authentication	24%
Missing authorization	11%

It is good at problems visible on one line: a query built from user input, a path taken from a request. It misses most authorization and authentication problems, because the check that should protect a route usually lives in another file. 269 of the labels are kinds Supercov does not check for at all, such as missing rate limits.

It also reports the most false positives on the benchmark’s decoy findings: 26, against 1 to 23 for the others.

By language

Language	Repositories	Supercov F1	Best F1
Python	32	0.54	0.82
TypeScript	27	0.36	0.65
JavaScript	13	0.34	0.74

Python is its strongest language. On JavaScript, many of its false positives are in bundled third-party libraries, such as copies of jQuery or Bootstrap, that it should skip.

When to use something else

You want the most findings and can wait. A Codex CLI agent or Kolega finds nearly twice as much. Run it before a release, not on every change.
You want low cost and can wait a few minutes. A DeepSeek V4 Flash agent is cheaper and more accurate, at about 2.6 minutes a repository.
You want fast feedback on every change. Supercov answers in seconds, so your agent can run it after each edit with npx supercov security patch.

The tools also combine well: Supercov on every change, an agent-based scan before a release.

How we measured

Benchmark: 72 of RealVuln’s 140 repositories, the half we did not tune the checks on, each at the commit its labels were written for. They are deliberately vulnerable Python, TypeScript and JavaScript apps, so results on production code will differ.
Supercov: version 2.0.1 with its default settings, one repository at a time on a laptop. Cost is Jev’s list price of $42 per billion input tokens.
Other scanners: their findings, times and costs are RealVuln’s own published runs, and their times come from different machines. We timed Semgrep ourselves, with the same version and settings, on the laptop that ran Supercov. GLM-5.3 had 3 failed runs, counted as finding nothing.
Scoring: a finding counts when it names the right file, a matching vulnerability type and a line within 10 of the label.

Download the results or everything needed to rerun it: Supercov’s report for every repository, the times and the scoring script. Checked September 25, 2026.