Supercov vs Semgrep for security scanning

The difference

Semgrep matches code against rules. It is free, runs offline and every result is an exact rule match, but it only finds what a rule describes. Supercov asks an AI model twelve security questions about each file, so it catches problems no rule names, for about 5¢ a repository.

Results

72 deliberately vulnerable repositories, 2,016 labelled vulnerabilities. Semgrep ran with --config auto, its default open-source rules. We timed both on the same laptop.

	Supercov 2.0.1	Semgrep, auto rules
F1	0.42	0.11
Finds	43%	9%
Right	42%	14%
Median time	10 s	8 s
Cost per repository	5¢	free

Both finish in seconds; Semgrep is a little faster, and free. Supercov finds about five times as much.

By language

F1	Supercov	Semgrep
Python, 32 repositories	0.54	0.08
TypeScript, 27	0.36	0.14
JavaScript, 13	0.34	0.12

With default rules, Semgrep finds few of these vulnerabilities. Rules written for your framework would do better; they were not part of this test.

Choose Semgrep when

you need a scanner with no API calls and no cost;
you want to enforce your own rules, like banning a function;
the same input must always give the same result.

Choose Supercov when

you want vulnerabilities found without writing rules;
5¢ a repository is fine for about five times the findings.

You can run both: Semgrep for the rules you already have, Supercov for everything else. See Accuracy, speed and cost for how this was measured.