Methodology
How we measureAI-built app security.
Every statistic on this site comes from ShipSafe’s own independent, read-only scans of publicly-deployed apps. We read the code and probe the live app from the public internet, and we only report data exposure that an actual anonymous request returned — never a guess.
On consent, and what we never do
When you verify your own app, you authorize the scan. The deep active checks (confirming a leaked key is actually live, probing your backend) run on your own machinevia our CLI or GitHub App, and we sign the result. We never point our servers at infrastructure you can’t prove you own. The studies below use only read-only, browser-equivalent requests to publicly-deployed apps: we never log in, never validate a credential against its provider, and never change anything. Independent verification, with consent built in.
The studies
Each headline figure, its sample size, and exactly what we count — cited, not asserted.
Deng, Fan & Meng (arXiv:2606.23130)
200 deployed apps, Claude Code and Lovable
90% contained at least one vulnerability two expert reviewers independently judged exploitable against the live site
Independent academic researchers. arXiv preprint, not yet peer-reviewed.
Deng, Fan & Meng (arXiv:2606.23130), Table 2
same 200 apps
75.5% had at least one Broken Access Control flaw (OWASP A01), against an OWASP baseline incidence of 3.74% across general web apps
All severities, not critical-only.
Pearce et al., IEEE S&P 2022
1,689 programs generated by GitHub Copilot
about 40% contained the weakness class the scenario was designed to test for
Lab-generated code, not deployed apps. 2021-era Copilot; a 2023 replication found the Python rate had fallen from 36.5% to 27.3%.
These are other people's studies, not ours. We used to publish our own per-builder rates here and pulled them: we could not produce the underlying data on request, and a verifier that cannot show its working has nothing to sell.
How a scan runs
Read-only, proven-not-inferred, and counted once per app.
Read-only and non-destructive: GET requests only, bounded, never writing or deleting.
Both surfaces: the code (or repo) for logic bugs, and the deployed app from the public internet.
Data exposure is proven, not inferred: a single anonymous read against a Supabase table or storage bucket, or a Firebase database, that actually returned rows.
Critical-severity follows the scanner's CWE/OWASP-mapped scale; a finding is counted once per app.
Scope and limits
What these numbers are — and, honestly, what they are not.
These are point-in-time, automated scans of a sample of apps. They describe how often a class of issue appears across AI-built apps, not the security of any single app. A passing scan is not a guarantee, a penetration test, or a compliance audit. New code introduces new risk, so re-scan after changes.
Want the same proof for your own app?
Independent, read-only, consent-built-in — and signed. See where your app lands in about two minutes.
— ShipSafe · Independent verification, with consent built in —