vulnerabilitydata.io
11 · Methodology

How this was built, and what to distrust

Sourcing

The site starts from outcomes. Nine report families (IC3, DBIR, IBM, Chainalysis, Coveware, Coalition, CrowdStrike, Unit 42 and M-Trends) supply the outcome figures, each edition read from its own publication and entered with its URL and the quoted span. The AI ledger re-reads each documented case from its primary document. Beneath them sit the vulnerability series (CVE.org and NVD annual totals, Zero Day Initiative's Patch Tuesday reviews, MITRE's CWE Top 25) and the live feeds described below. Every figure traces to a named, dated source; where two sources disagree, both are kept, and two independent sources were sought for each annual CVE total.

What we corrected

On 2026-10-20 four independent audits recomputed the live series from their feeds, checked every chart against its data, opened every cited source, and attacked every interpretation. What they found and what changed:

  • Patch Tuesday was re-read month by month against each release's own ZDI review, under one counting rule (new Microsoft CVEs, third-party re-documented CVEs excluded). Seventeen months changed; July 2026 is 621 (an earlier version said 569) and June 2026 is 208 (it said 200).
  • The SEC Item 1.05 series counted 8-K/A amendments, against its own stated method: 85 filings became 59.
  • The leak-site comparison set a fully backfilled year-ago window against a recent window still filling in. Both windows now count only victims posted within 30 days of month end, and the cash-out conjecture's status changed from consistent to refuted. Its condition was not changed.
  • Have I Been Pwned counts now exclude stealer logs and malware dumps.
  • The ~5-day time-to-exploit figure is Mandiant's 2023 average, not a median; the page said median. The ~32% NVD enrichment figure was not in its cited source and was replaced with NIST's own number. Figures no reachable source supports were removed and are listed in each data file's list of gaps.

Known limitations

  • Monthly CVE granularity is thin — most trackers render counts in JavaScript. Annual figures are solid; intra-year monthly series are partial.
  • 2016–2021 annual totals rest largely on one renderable source (cvedetails), counted by CVE-ID year; marked medium-confidence in the data.
  • Zero-day-in-the-wild counts are Google/Mandiant telemetry, self-revised each year — not independently corroborated.
  • No dataset cleanly isolates an AI-attributable effect from structural confounders. So no outcome here is labelled as caused by AI. An outcome figure carries one of three labels: AI: not attributed when its source records no AI field, AI: attributed by the source with the note that it is a lower bound, or AI: asserted by the source, not measured when a report credits AI in words. Each documented AI case in the ledger carries its evidence grade instead (listed below).
  • Breach data carries its own traps, kept explicit on the Breaches page: ITRC changed methodology in 2020 (breaches/records → compromises/victim notices — the victim metric is not continuous and is never charted across the break); ITRC silently restates prior-year totals by a few events between editions (this site uses as-originally-reported figures, restatements noted in the data file); DBIR counts track its changing contributor base, not world breach volume (vector percentages are the trend signal, raw counts are not); DBIR editions before 2021 and IBM's 2016 figure were omitted rather than published unverified.

The hard limits on attributing outcomes to AI

Three limits hold for every page, stated once here.

  • No breach can be tagged AI-caused from the victim's side. A phishing email, an exploit or a ransom note does not record whether a model wrote it, so complaints, breach notices and filings can only show what a victim noticed. An AI tag that depends on noticing is a lower bound, never a share.
  • Open-weight use leaves no provider telemetry. A model run on the attacker's own hardware reports to no one; it appears only when someone takes malware or an intrusion apart.
  • The counterfactual is never observed. No one sees the same year without AI, so every estimate of AI's effect rests on an assumption, and each registered test names its assumption and its confounders.

Because of these, a change in an outcome series is credited to AI only when that series' own source attributes it, never by timing or adjacency; counts by kind of AI tool are shown as what each observer could see, never as prevalence.

Evidence grades

Each row in the AI-involvement ledger carries one grade, from a closed list, with the quoted span it rests on:

  • independently confirmed: a party other than the model provider and the attacker confirmed it, such as an incident responder or a researcher with the sample.
  • confirmed by an authority or the victim: a government, CERT, court or the victim confirmed it.
  • self-reported by the model provider: the model provider reports misuse of its own models, with no outside confirmation in the document.
  • research demonstration: a capability shown in a lab, a competition or a proof of concept, not an incident.
  • asserted: claimed without evidence a reader can check.

A language model reads each row's quoted span and assigns its grade, tool class, attack stage and outcome against the definitions above. No person has yet checked those labels against the documents, so every row links its document and shows its span. Each row is also marked as an attack, a defensive tool or a demonstration; only attacks are counted as incidents, and an attack campaign reported by several observers counts once. Standing findings on the Analysis page carry their own grade from a fixed rule: estimated with bounds only when a registered test has a final reading and an interval is computed and shown; a point estimate, no interval computed when a registered test has a final reading but no interval; documented when a source itself attributes a figure to AI; and not yet measurable otherwise, each with the instrument that would settle it. A source that credits a change to AI in words, without measuring how much, is shown as that source's assertion, quoted, never as a measured share.

The documents the AI ledger reads

The attack count reads 14 published documents: 7 model providers' reports (OpenAI, Anthropic and Google GTIG), a victim's own statement (Arup), 2 security researchers' reports (HP Wolf Security and Zenity Labs), a CERT advisory (CERT-UA) and 3 incident-response and threat-intel reports (Wiz, Palo Alto Networks Unit 42 and Cisco Talos). It holds what those documents describe, so it is a partial record, not a census of AI-assisted attacks. With defensive tools and demonstrations included, the ledger reads 22 documents: 10 model providers' reports (OpenAI, Google Project Zero, Google, Anthropic and Google GTIG), a victim's own statement (Arup), 6 security researchers' reports (HP Wolf Security, XBOW, ESET, Mozilla and Zenity Labs), a CERT advisory (CERT-UA), a competition report (DARPA) and 3 incident-response and threat-intel reports (Wiz, Palo Alto Networks Unit 42 and Cisco Talos). A case no document in this set describes is not in the ledger, so every count drawn from it is a floor on what was published, and the set grows as documents are added.

Changes in how sources count

Every dated change in how a source counts is listed in the dataset with its primary source. A change is drawn as a dashed seam only on a series whose counting it changed; each entry below says which chart draws it, if any. IC3's business email compromise chart also carries seams where IC3 redefined the category.

  • 2016-11 · CVE Numbering Authority expansion begins. CVE.org's own CNAs-added metric shows 0 CNAs added in all of 2015, then 12 in November 2016 and 7 in December 2016 (23 in 2016, 30 in 2017, summed from the monthly values). More organizations issuing CVE IDs raises the CVE count independent of how many flaws exist, which is the counting change behind the site's existing 2017 seam on the CVE-per-year charts. Source Drawn on the annual CVE charts (Overview and CVE Data).
  • 2020 · ITRC switches its impact measure from records exposed to individuals impacted. ITRC's 2019 report measured "sensitive records exposed"; its 2020 report measures "individuals impacted". The two are different units and cannot sit on one line. Separately, from the 2021 report ITRC labels its event count "data compromises" and gives 2020 as 1,108, the same number the 2020 release called "data breaches", so the releases opened here show a relabel of the count, not a different 2020 count. Whether the definition behind the count widened is not stated in these releases (the full reports are gated). Source Drawn on the ITRC charts (Breaches and Combined).
  • 2021-11-03 · CISA creates the Known Exploited Vulnerabilities catalog (BOD 22-01). The KEV catalog starts on this date with a backfill of vulnerabilities exploited years earlier, so 2021 KEV additions and any CVE-to-KEV gap for older CVEs measure the catalog's launch, not exploitation speed. The time-to-exploit series on this site counts only CVEs published on or after this date for that reason. Source (no chart here draws it: the series it changes is not charted here, or not across that date)
  • 2023-05-27 · Cl0p's MOVEit Transfer mass exploitation begins. One campaign against one file-transfer product stole data from many organizations at once from late May 2023; each victim can then appear separately as a notice, a leak-site post or a breach-report case. Verizon's 2024 DBIR names MOVEit as a driver of its near-tripling of exploitation as a breach entry point. Series covering 2023 data can carry a one-campaign bulge. ITRC's 2023 release says zero-day attacks jumped and supply-chain victims surged but does not name MOVEit. Source Drawn on the monthly strips on Live.
  • 2023-12-18 · SEC Form 8-K Item 1.05 disclosures become mandatory. Before this date there was no Item 1.05, so the series starts here by construction; any count before it is zero because the item did not exist, not because there were no material incidents. The rule was adopted July 26, 2023 and effective September 5, 2023, but incident disclosure was not required until this compliance date. Source Drawn on the monthly strips on Live.
  • 2023-12-19 · FBI disrupts ALPHV/BlackCat and seizes its websites. A law-enforcement action against a group the DOJ calls the second most prolific ransomware-as-a-service variant by ransoms paid, including seizure of its websites. Leak-site victim counts and distinct-group counts around this date can reflect what the tracker could see after the seizure, not only how many organizations were attacked. Source Drawn on the monthly strips on Live.
  • 2024-02-15 · NVD posts its analysis-delay notice; enrichment slows. NIST began falling behind on enriching new CVEs (severity scores, product lists, weakness types). From here, any NVD-derived field on recent CVEs is missing for a growing share of records, while the CVE record count itself (from CVE.org) is unaffected. NIST later described the period as a processing slowdown in spring and early summer of 2024. Source (no chart here draws it: the series it changes is not charted here, or not across that date)
  • 2024-02-20 · Operation Cronos takes over LockBit's leak site. The NCA took control of LockBit's administration environment and its public leak site; the NCA calls LockBit the world's most harmful cyber crime group. Leak-site victim and group counts from this month onward can reflect the takedown as well as attack volume. Source Drawn on the monthly strips on Live.
  • 2024-05-01 · DBIR redefines its human-element metric. From the 2024 edition, Verizon removed malicious insiders (Privilege Misuse) from the human-element share. The old definition would have given 76% against the new 68%, so human-element shares before and after the 2024 edition are different measures. Source (context only, not drawn)
  • 2024-05-01 · DBIR adds a third-party involvement metric. The share of breaches involving a third party is new in the 2024 edition, so there is no like-for-like value before it. The 2026 edition's 48% third-party figure can only be compared back to the 2024 edition. Source (context only, not drawn)
  • 2024-05-21 · SEC staff statement steers non-material incidents from Item 1.05 to Item 8.01. The Director of Corporation Finance asked companies to file incidents they judged immaterial, or had not yet judged, under Item 8.01 rather than Item 1.05. Item 1.05 counts after this date cover a narrower set of filings than before it, so a fall in Item 1.05 filings after May 2024 can be a filing-practice change. The Item 8.01 cybersecurity series is the other half and carries this marker too. Source Drawn on the monthly strips on Live.
  • 2024-06-15 · Smaller reporting companies come under Item 1.05. Smaller reporting companies had 180 extra days, so the population that must file Item 1.05 widens on this date. A rise in filings after mid-June 2024 can come from more companies being covered. Source Drawn on the monthly strips on Live.
  • 2025 · IC3 adds an "AI Related" descriptor (AI-tagged figures only). The 2025 IC3 report adds an "AI Related" descriptor and marks earlier years as not captured. A descriptor is a tag IC3 adds to a complaint after its crime type is chosen, so it changes nothing about how complaints or losses are counted: total complaints, total losses, business email compromise and every other crime type are counted as before, and no seam belongs on those charts. It affects only the AI-tagged figures, which count complaints whose text mentions AI. Those depend on the victim knowing and saying AI was involved, so they are lower bounds with no earlier baseline. Source (no chart here draws it: the series it changes is not charted here, or not across that date)
  • 2025 · Most breach notices no longer say how the attacker got in. ITRC reports that 70 percent (2,324) of 2025 data breach notices did not include attack information, against 65 percent (2,049) in 2024 and 45 percent (1,449) in 2023, and that in 2020 nearly every notice did. The breach count is unaffected, but any breakdown of breaches by how they happened now rests on a shrinking minority of notices, so it cannot show a change in attack method, AI-driven or otherwise. Source (context only, not drawn)
  • 2026-03-31 · CVE Program reaches 500 CNAs. CVE.org's own running count went from 435 CNAs (2025-01-07) to 484 (2025-12-09) to 500 (2026-03-31) and 559 (2026-10-20). Growth is gradual, so no break is drawn, but more issuers keep raising CVE volume independent of the number of flaws. Source (context only, not drawn)
  • 2026-04-15 · NIST stops enriching most new CVEs by default. NVD now enriches only CVEs in KEV, in federal software, or in EO 14028 critical software; the rest are listed as lowest priority. Backlogged CVEs published before March 1, 2026 moved to "Not Scheduled", and NIST no longer adds its own severity score where the CNA gave one. Any NVD-derived share (CVSS, CPE, CWE from NVD) after this date covers a selected subset of CVEs. Source (no chart here draws it: the series it changes is not charted here, or not across that date)
  • 2026-06-10 · BOD 26-04 replaces BOD 22-01; KEV inclusion criteria stay the same. CISA revoked BOD 22-01 and issued BOD 26-04, which ties federal remediation timelines to KEV status, exploit automation, technical impact and exposure. CISA states the criteria for adding a vulnerability to KEV did not change, so this is listed to stop a reader reading a governance change into the KEV series; no break is drawn. Source (context only, not drawn)
  • 2026-07-28 · CVE Program lets Anthropic and OpenAI assign CVE IDs for what they find. A closed six-month pilot lets two frontier AI labs issue CVE IDs for vulnerabilities they discover in widely used products outside another CNA's scope. From this date a CVE found by either lab can be published with the lab as its numbering authority, so the monthly count of CVEs whose record credits an AI tool can rise because of who is allowed to file, with no change in how much AI finds. Whether the pilot adds CVEs that would not otherwise have been assigned is not stated. Source Drawn on the AI-credited CVE chart on The AI Question.
  • 2026-07-27 · GitHub gates its public bug bounty on HackerOne signal, citing AI-generated reports. From this date GitHub's public program limits submissions from researchers below a HackerOne signal threshold and cuts public payouts, while a new invite-only tier pays more. Counts of valid or AI-generated reports on GitHub's program can fall because of who may submit, with no change in what AI finds. Source (context only, not drawn)
  • 2026-05 · Apple can pause a researcher's reports for 180 days over unvalidated AI findings. Apple's bounty guidelines add a 180-day processing pause for repeated ineligible reports, naming AI-discovered issues submitted without validation. Valid-report and payout counts on Apple's program can change because of who keeps submitting. Source (context only, not drawn)
  • 2026-10-22 · Ledger source list adds Mozilla’s two reports of AI-found Firefox bugs. Two defense documents joined the ledger’s source list after the hypotheses were registered: Mozilla’s March 2026 account of Anthropic’s Frontier Red Team findings and its April 2026 account of Claude Mythos Preview. Both are defense rows dated before 2026-10-22, so they enter the baseline of any comparison of ledger rows by date, with no change in what AI did. Source (context only, not drawn)

Stance

This site is neutral about AI's net effect: it does not declare whether AI favors attackers or defenders. It reports what the outcome series (losses, ransom paid, attack speed, detection, breach counts) and the exploitation and vulnerability series beneath them show; it credits a change to AI only where that series' own source does; it grades every documented AI case by its evidence and keeps defensive tools and demonstrations apart from attacks; and it marks what no source can yet see. The data files behind the charts are listed on Sources, each marked with whether it is public in Substrate.

The live series

The Live page is rebuilt from public feeds by collectors in Substrate, run weekly: seven public feeds (CISA KEV, CVE.org records, ransomware.live, SEC EDGAR (8-K Items 1.05 and 8.01), Have I Been Pwned, the California Attorney General's breach list and HHS OCR's breach portal), drawn as eight monthly series. None needs an API key. The same run counts AI-credited CVEs from CVE.org records against a published, versioned tool list.

  • Time to exploit is the days from a CVE's CVE.org publication to CISA adding it to KEV, over CVEs published on or after KEV's launch (2021-11-03), grouped by the month CISA listed them. The cutoff keeps out the catalog's launch backfill of years-old bugs. Because CISA lists a bug only once exploitation is confirmed, the figure is an upper bound on the real time to exploit, and monthly medians rest on as few as 3 and at most 43 entries, so single months move a lot.
  • Leak-site victims are the records ransomware.live returns for each month. They show victims who were posted, which skews toward those who did not pay.
  • SEC 8-K Item 1.05 counts 8-K filings that EDGAR tags with item 1.05, found through full-text search for "Item 1.05" and deduplicated by accession number. Amendments are excluded. Materiality is each company's own call, and since a May 2024 SEC staff statement, incidents not judged material are steered to Item 8.01, which this series does not count.
  • SEC 8-K Item 8.01 cyber incidents are counted as their own series on Live: filings under Item 8.01 found by a phrase screen for a company's own cybersecurity incident, whose precision has not been checked against labels. They are never added to the Item 1.05 count.
  • California Attorney General notices and HHS OCR reports (breaches of US health data affecting 500 or more people) are counted by the month each was submitted, not the month of the breach.
  • Have I Been Pwned counts verified breaches by the month they were loaded into the service, not the month they happened.
  • The breach series are never summed. They count different populations, and adding them would double count.

The rolling-window conjectures on the Analysis page were written with their refutation conditions on 2026-10-20. Their windows include months already published then, so only readings of later months test them in advance. The three release event studies were registered retrospectively on 2026-10-22, after their windows had passed, with dates and thresholds frozen for every later reading. Every status is computed by code, never judged by hand.

The standing analysis

The Analysis page re-asks the same questions across every dataset — are breaches rising because of AI, what mechanisms the data supports, and what it can't answer. The site will not build while any data file has no section in the analysis. That check covers coverage only: keeping each number in the analysis current still takes review.