# Security scanners compared: accuracy, speed and cost

> Supercov, Codex CLI, Claude Code, Kolega, DeepSeek, GLM and Semgrep on 72 vulnerable repositories: how much each finds, how fast, and what it costs.

Web page: https://supercov.com/compare/security · Published 2026-09-25
All Supercov pages for agents: https://supercov.com/llms.txt · Full text: https://supercov.com/llms-full.txt

## The short version

- **Most accurate:** Codex CLI with GPT Daybreak Blue (F1 0.71) and Kolega
  DevSec Max (finds 91%). Several minutes and about $4 a repository.
- **Cheapest AI scanner:** a DeepSeek V4 Flash agent. 2¢ a repository, F1 0.48,
  2.6 minutes.
- **Fastest AI scanner:** Supercov. 10 seconds and 5¢ a repository, F1 0.42.
- **Rules only:** Semgrep's default rules find 9% of these vulnerabilities, in
  8 seconds for free.

## Results

Each scanner ran on the same 72 deliberately vulnerable repositories, with
2,016 labelled vulnerabilities between them.

| Scanner | F1 | Finds | Right | Speed | Cost per repository |
| --- | ---: | ---: | ---: | --- | ---: |
| Codex CLI, GPT Daybreak Blue | 0.71 | 82% | 63% | Slow, 5.4 min | $3.78 |
| Codex CLI, GPT-5.6 Sol | 0.70 | 79% | 62% | Slow, 8.8 min | $3.87 |
| Kolega DevSec Max 0.1.0 | 0.69 | 91% | 56% | not recorded | not recorded |
| DeepSeek V4.1 Flash agent | 0.51 | 50% | 53% | Minutes, 1.6 min | not recorded |
| GLM-5.3 agent | 0.51 | 54% | 48% | Slow, 7.9 min | 74¢ |
| Claude Code, Sonnet 5 | 0.49 | 44% | 57% | Minutes, 2.8 min | 95¢ |
| Codex CLI, GPT-6 Astra | 0.49 | 54% | 45% | Slow, 6.4 min | $6.99 |
| DeepSeek V4 Flash agent | 0.48 | 41% | 59% | Minutes, 2.6 min | 2¢ |
| DeepSeek V4 Pro agent | 0.46 | 37% | 63% | Minutes, 3.8 min | 7¢ |
| **Supercov 2.0.1** | **0.42** | **43%** | **42%** | **Instant, 10 s** | **5¢** |
| Semgrep, auto rules | 0.11 | 9% | 14% | Instant, 8 s | free |

**Finds** is the share of labelled vulnerabilities found (recall). **Right** is
the share of findings that were real (precision). **F1** combines the two.
Speed is the median time per repository: instant is under a minute, slow is
over five.

## Pick by what you need

| You want | Pick |
| --- | --- |
| The most findings before a release | Codex CLI with GPT Daybreak Blue, or Kolega |
| An AI review for pennies, and a few minutes to wait | A DeepSeek V4 Flash agent |
| A check your agent can run after every change | Supercov |
| Known patterns, free and offline | Semgrep, with rules for your stack |

Most teams will want two: something fast on every change, and a thorough scan
before a release.

## Compared one by one

- [Supercov vs Codex CLI](https://supercov.com/compare/security/codex.md)
- [Supercov vs Claude Code](https://supercov.com/compare/security/claude-code.md)
- [Supercov vs Kolega](https://supercov.com/compare/security/kolega.md)
- [Supercov vs DeepSeek agents](https://supercov.com/compare/security/deepseek.md)
- [Supercov vs Semgrep](https://supercov.com/compare/security/semgrep.md)

## How we measured

The repositories are the 72 in the half of
[RealVuln](https://github.com/kolega-ai/Real-Vuln-Benchmark) we did not tune
Supercov on. They are Python, TypeScript and JavaScript apps written to be
vulnerable, so expect different numbers on production code. Supercov 2.0.1 ran
with default settings on a laptop. Every other scanner's findings, time and
cost come from RealVuln's published runs, made on other machines, except
Semgrep's time, which we measured on the same laptop. Kolega builds RealVuln.

[Accuracy, speed and cost](https://supercov.com/docs/security-benchmark.md) has the details: what
Supercov finds by vulnerability type, results by language, and how to rerun it.


Evidence: [every scanner's results (JSON)](https://supercov.com/downloads/security-benchmark-2026-09-25.json) · [Supercov's reports, timings and scoring script (ZIP)](https://supercov.com/downloads/supercov-security-benchmark-2026-09-25.zip). Measured September 25, 2026 on 72 RealVuln repositories.