Cheap to find, expensive to check: what curl, NIST and DARPA each did about it

Between April 2019 and January 2026, the curl project's bug bounty confirmed 87 real vulnerabilities and paid out more than 100,000 US dollars. Daniel Stenberg, who maintains curl, is proud of that record and says so. He also closed the programme. In the years when it worked, somewhere north of 15 percent of submissions turned out to be genuine. From the start of 2025 the confirmed rate fell below 5 percent. Not one report in twenty was real, and each of the other nineteen still had to be read, understood and disproved by a human being.

Stenberg gave three reasons in his announcement: AI-generated submissions, a drop in quality even among reports that were not obviously machine-written, and a growing tendency for reporters to argue a finding upward rather than help fix anything. The programme now accepts no reports through HackerOne and pays nothing at all.

Three months later, a considerably larger institution narrowed what it was willing to judge as well.

The pattern, stated plainly

In April 2026, NIST changed how the National Vulnerability Database works. CVE submissions rose 263 percent between 2020 and 2025, and the first quarter of 2026 came in nearly a third higher than the same period the year before. NIST enriched close to 42,000 CVEs in 2025, which was 45 percent more than any prior year, and it was not enough. So the goal of analysing every CVE was abandoned. From 15 April, NIST prioritises three categories: entries in CISA's Known Exploited Vulnerabilities catalog, software used by the federal government, and critical software as defined by Executive Order 14028. Everything else is listed and labelled "Lowest Priority, not scheduled for immediate enrichment." The existing backlog, anything published before 1 March 2026, was moved to "Not Scheduled." NIST also stopped routinely issuing its own severity score where the submitting authority had already provided one.

Eight months before that announcement, DARPA closed the AI Cyber Challenge. Seven autonomous systems analysed more than 54 million lines of code. They found 86 percent of the competition's synthetic vulnerabilities, up from 37 percent at the semifinals a year earlier, and patched 68 percent of what they found, up from 25 percent. Average time to submit a patch was 45 minutes. Average cost per competition task was about 152 dollars. The teams also turned up 18 real vulnerabilities nobody had planted, and wrote 11 patches for them. Four of the seven systems were open sourced immediately.

Three events, eight months end to end, and they are normally filed as three different kinds of story: a maintainer's burnout, an institution running out of road, a technical triumph. I think they are one story, and the variable that connects them is not the technology. It is what each institution asked people to prove.

The easy reading is wrong, and curl's own data show it

The tempting summary is that AI-generated noise is overwhelming security. It is tempting because it is partly true and because it flatters everyone who already distrusts the tooling. It also does not survive contact with the evidence.

Stenberg asked HackerOne for comparative numbers, and published them. Over four quarters, curl's inbound report volume rose sharply while other bounty-paying open source programmes in the same cohort, including Ruby, Node and Rails, stayed flat or declined slightly. If cheap generation were simply flooding open source security, those lines would move together. They did not. Stenberg's own conclusion is unusually honest for a person with every reason to blame the obvious culprit: he suspects the money is a large part of it, and writes that why curl suffers more than its peers "remains a subject for further speculation and research."

So the mechanism is narrower and more interesting than a flood. Where a payout exists, the cost of producing something that resembles a finding collapsed, while the cost of establishing whether it is true did not move at all. That asymmetry is the whole problem. A report is cheap to write and expensive to refute, and it is expensive to refute in a way that scales with the reader's attention rather than the writer's.

What DARPA required that a bounty inbox does not

Set the AIxCC result next to curl's inbox and the difference is not the models. Broadly the same generation capability was on both sides. The difference is that the competition had a ground truth and required systems to demonstrate against it. DARPA's own account credits the winning team with performing best at finding and proving vulnerabilities, pairing each vulnerability with a patch, and scoring on the rate of accurate submissions. A system that produced a confident, wrong report scored nothing. In a bounty inbox, a confident wrong report costs the sender nothing and costs the maintainer an afternoon.

That gives a rule worth stating on its own, because it generalises well beyond security: automation scales cleanly wherever verification is cheap, and it loads the bottleneck wherever verification is expensive. Screening, sorting, matching and generating candidates are all verification-cheap operations, and machines are now very good at them. Deciding whether a candidate is true is verification-expensive, and nothing about the last two years has made it less so. Pointing more generation at an evaluative bottleneck does not relieve it. It fills it faster.

Note that this is not an argument that the tooling is bad. AIxCC is the strongest evidence available that these systems can meet a real evidentiary bar, and they met it at 152 dollars a task while bug bounties for the same class of work run from hundreds to hundreds of thousands. The point is that they met it because the bar was there.

The strongest objection

The best counterargument is that NIST did not retreat, it triaged, and triage was overdue.

The data behind that objection are good. The inaugural Cyentia and FIRST study of EPSS, covering 237,687 published CVEs and more than 8.6 million daily observations of exploitation activity, found that only about 6 percent of all published vulnerabilities have ever been observed being exploited in the wild. Half of known exploited CVEs are never seen in more than 0.02 percent of organisations. Prioritising by exploitation probability rather than severity reaches 93 percent risk coverage where a CVSS 9-and-above strategy reaches 37 percent for the same effort. On that evidence, enriching every CVE to the same standard was never a good use of scarce analysts, and NIST spending its capacity on the exploited slice is straightforwardly the right call.

I think that objection is correct as far as it goes, and it does not rescue the position, for two reasons the same study supplies.

The first is that prioritisation is not evaluation. KEV and EPSS are observational and downstream. They answer "is this being attacked now", which is a question that can only be answered after somebody has been attacked. They are silent by construction about a vulnerability nobody has tried yet, which is precisely the category that cheap automated discovery produces at volume. Using exploitation signal to decide what to analyse works well while the population of unanalysed findings is stable. It is a weaker guide when that population is the thing growing.

The second is that the residual is not empty, and the study says so. Prioritising everything at an EPSS score of 0.1 or above achieves 80 percent coverage, which means one exploited vulnerability in five sits below the line. And 38 percent of current exploitation attempts target vulnerabilities published more than ten years ago, which is close to a description of the backlog that has just been moved to "Not Scheduled." The triage is defensible as a way of allocating a fixed number of analyst hours. It is not evidence that the unenriched remainder does not matter, and it should not be read as one.

What actually happened is that an unaffordable universal standard was replaced with an affordable partial one, and the difference is now carried privately, by vendors who sell enrichment. That is a real change in who can afford to know things, and it happened without much argument because it arrived dressed as an operational notice.

Where this argument could be wrong

Three places, and they are worth naming rather than burying.

NIST attributes the surge in submissions to submission volume, not to AI, and nowhere connects it to automated discovery. The link between cheap generation and NVD's capacity problem is my inference from the timing and the shape of the pressure, not a claim anyone at NIST has made. Second, curl is confounded by the bounty money, by Stenberg's own account, so it demonstrates the mechanism rather than its prevalence, and its peers genuinely were not affected the same way. Third, a scored competition on 63 planted vulnerabilities is a controlled setting, and controlled settings flatter the systems that run in them. Eighteen real vulnerabilities discovered against 54 million lines of code is a real result, but it is a modest one next to the synthetic scoreboard, and DARPA reported both, to its credit.

The rule that follows

If the bottleneck is evaluation, then the useful intervention is not more discovery and not better filters bolted on afterwards. It is to move the evidentiary standard to the point of submission, so the cost of making a claim rises with the cost of checking it.

Ask for the artefact, not the assertion. A crash input that actually triggers the bug. A patch. A reproduction. Something that a machine on the receiving end can check without a person reading prose and forming a judgment about the sender's sincerity. curl's own experience supports this from the other direction: Stenberg notes that pull requests have not been a problem for the project, because a pull request runs against tests and scanners and consumes no human attention until it passes them. Same project, same era, same tooling available to contributors, opposite outcome, and the difference is that one channel demanded something checkable and the other accepted a paragraph.

That is not a security lesson. It is what every evidence pipeline is about to have to learn, and the fields that already require the artefact rather than the claim, code that must compile, data that must be deposited, a proof that must verify, are going to find the next few years considerably easier than the fields that trade in assertions.