LLM Testing Statistics 2026: Adoption, Hallucination and Eval Benchmarks

This is a reference page. Every figure below comes from a named, publicly available source, with the publisher and year stated next to it. Nothing here is our own data, nothing is estimated, and nothing is included that we could not open and read directly. Where a source is gated behind a download, we have left the figure out rather than repeat a number from a search snippet.

Figures last verified: 6 September 2026.

One theme runs through all of it. Adoption of generative AI inside engineering organisations is essentially finished as a story. Measurement of it has barely started. Every dataset below shows the same gap between what teams ship and what teams test, and the security data shows what falls into that gap.

GenAI adoption in engineering organisations

Adoption numbers have converged. Across four independent surveys with different populations and methods, the answer is the same: almost everyone is using this, and a much smaller group has anything in production that they would call an agent.

• 88% organisational adoption of AI, the headline adoption figure for the year, per the Stanford HAI AI Index Report 2026 (Stanford Institute for Human-Centered AI, 2026).
• 90% of respondents report using AI at work, from a survey of nearly 5,000 technology professionals, per the 2025 DORA report (Google Cloud / DORA, published 24 September 2025).
• More than 80% of those DORA respondents believe AI has increased their productivity, which is the belief figure and not a measured throughput figure (Google Cloud / DORA, 2025).
• 84% of developers are using or planning to use AI tools in their development process, from roughly 49,000 respondents, per the 2025 Stack Overflow Developer Survey (Stack Overflow, 2025).
• 51% of professional developers use AI tools daily (Stack Overflow, 2025).
• 51% of organisations are using agents in production today, rising to 63% among companies with 100 to 2,000 employees, per the State of AI Agents report (LangChain, published 12 June 2026, over 1,300 professionals surveyed).
• 78% of those respondents have active plans to put agents into production soon (LangChain, 2026).
• 16% of enterprise deployments and 27% of startup deployments qualify as true agents rather than simpler assisted workflows, per 2025: The State of Generative AI in the Enterprise (Menlo Ventures, published 9 December 2025, ~500 US enterprise decision-makers surveyed 7 to 25 November 2025).
• 47% of AI deals go to production, against 25% for traditional SaaS (Menlo Ventures, 2025).
• 76% of AI use cases are purchased rather than built internally, up from 53% in 2024 (Menlo Ventures, 2025). Buying rather than building shifts the testing burden onto vendor evaluation, which is not a burden most QA functions are set up for.

Trust, and the gap between using AI and believing it

The interesting number in every developer survey is not adoption. It is the distance between how many people use these tools and how many people trust the output they are shipping.

• 46% of developers actively distrust the accuracy of AI tools, against 33% who trust it (Stack Overflow, 2025).
• 66% of developers report encountering AI solutions that are almost right, but not quite - the failure mode that costs review time rather than saving it (Stack Overflow, 2025).
• 30% of DORA respondents report little or no trust in the code generated by AI, while 90% of the same population uses AI at work (Google Cloud / DORA, 2025).
• 52% of developers either do not use agents or stick to simpler AI tools, and 38% have no plans to adopt them (Stack Overflow, 2025).

DORA’s 2025 analysis also found that AI adoption showed a positive relationship with software delivery throughput and product performance, but continued to have a negative relationship with software delivery stability (Google Cloud / DORA, 2025). More output, less stable output. That is a testing problem, not a model problem.

Hallucination and eval benchmark findings

Hallucination is the statistic most often quoted without a benchmark attached, which makes most quotes useless. The numbers below are all tied to a stated task and a stated evaluator.

Vectara’s Hallucination Leaderboard measures summarisation faithfulness: models are given a source document and scored on whether the summary they produce is supported by it. It uses HHEM-2.3, Vectara’s commercial hallucination evaluation model, over a curated set of more than 7,700 articles across news, technology, science, medicine, legal, sports, business and education, covering over 100 models. Last updated 11 May 2026.

• The best-scoring entries sit between 1.8% and 3.3% hallucination rate, meaning fewer than 1 in 30 summaries contained unsupported content (Vectara, 2026).
• 3.1% hallucination rate with a 100% answer rate for openai/gpt-5.4-nano-2026-03-17, one of the strongest combinations on the board because it refuses nothing and still stays faithful (Vectara, 2026).
• 4.1% for meta-llama/Llama-3.3-70B-Instruct-Turbo, showing that an open-weights model can sit in the same band as frontier commercial models on this task (Vectara, 2026).
• 7.0% for google/gemini-2.5-pro and 7.8% for google/gemini-2.5-flash (Vectara, 2026).
• 10.9% for anthropic/claude-opus-4-5-20251101 and 10.3% for anthropic/claude-sonnet-4-20250514, a reminder that a more capable model is not automatically a more faithful summariser (Vectara, 2026).
• 23.3% for openai/o3-pro at a 100% answer rate, which is roughly one unsupported summary in four (Vectara, 2026).
• 24.2% for mistralai/ministral-3-3b-2512 at a 74.3% answer rate, so it both refuses more often and errs more often when it does answer (Vectara, 2026).

Domain-specific tasks look worse than the general leaderboard, which is exactly why a generic benchmark cannot stand in for your own eval set.

• More than 17% of responses contained incorrect information from Lexis+ AI and Ask Practical Law AI, tested on over 200 open-ended legal queries, per Stanford RegLab and HAI research (Stanford University, published 23 May 2024).
• More than 34% hallucination rate for Westlaw AI-Assisted Research on the same benchmark (Stanford RegLab / HAI, 2024). Both products are retrieval-augmented and marketed as hallucination-resistant.

Capability benchmarks moved in the opposite direction over the same period, which is why testing has to keep re-baselining.

• On SWE-bench Verified, a coding benchmark, model performance rose from 60% to near 100% in a single year (Stanford HAI AI Index Report 2026).
• 53% population adoption within three years for generative AI, faster than the PC or the internet (Stanford HAI AI Index Report 2026).

LLM incident and reliability data

Incident data is thinner than adoption data, because most organisations do not publish theirs. The two credible public series are the AI Index incident count and the penetration testing corpora.

• 362 documented AI incidents, up from 233 in 2024 (Stanford HAI AI Index Report 2026).
• 32% of AI and LLM findings are rated High Risk, against a much lower rate for findings overall, per the State of Pentesting Report 2026 (Cobalt, 2026, drawing on over 16,500 pentests across nearly 3,000 organisations over five years plus a survey of 450 information security professionals).
• 2.7x the rate at which high-risk findings appear in AI and LLM tests compared with the baseline (Cobalt, 2026).
• 38% resolution rate for AI vulnerabilities, the lowest remediation follow-through of any finding class Cobalt reports (Cobalt, 2026). Teams are finding AI defects and then not fixing them.
• 11% of software companies have had an AI or LLM security incident, per State of Pentesting in the Software Industry (Cobalt, published 30 July 2026).
• 9% of financial services and insurance organisations have had an AI or LLM security incident, the lowest of any sector, per State of Pentesting in Financial Services and Insurance (Cobalt, published 18 August 2026, sector sample n=65).

The security taxonomy those findings map to is the OWASP Top 10 for LLM Applications (OWASP GenAI Security Project, 2025 version), whose categories are Prompt Injection, Sensitive Information Disclosure, Supply Chain, Data and Model Poisoning, Improper Output Handling, Excessive Agency, System Prompt Leakage, Vector and Embedding Weaknesses, Misinformation, and Unbounded Consumption.

Eval and observability tooling adoption

This is where the numbers get uncomfortable. More teams have agents in production than have any structured way of scoring them.

• 39.8% of organisations use offline evaluation and 32.5% use online evaluation, against 51% who already run agents in production (LangChain, 2026).
• Performance quality was cited as the dominant barrier to agent adoption, described as more than twice as significant as cost or safety, and rising to 45.8% among small companies (LangChain, 2026). The barrier teams name is exactly the thing evaluation measures.
• 51% of technology company respondents use two or more control methods on their agents, against 39% in other sectors (LangChain, 2026).
• 83% of software companies run regular AI security assessments and pentests, the highest of any sector (Cobalt, 2026).
• 64% of software companies run a programmatic pentesting cadence, against 53% across all industries (Cobalt, 2026).
• 48% of financial services and insurance organisations test their AI applications programmatically, the highest of any sector, against a 39% average across industries (Cobalt, 2026).

Cost trends

Two things are happening at once and they get confused constantly. The price of a fixed capability is falling fast. Total spend is rising faster.

• 40x per year decline in the price to achieve GPT-4 level performance on a set of PhD-level science questions, per LLM inference price trends (Epoch AI, 2025).
• 9x to 900x per year is the range of decline rates across six benchmarks, so any single “prices fall Nx per year” claim depends entirely on which capability you hold fixed (Epoch AI, 2025).
• USD 37 billion spent by companies on generative AI in 2025, up from USD 11.5 billion in 2024, a 3.2x year-over-year increase (Menlo Ventures, 2025).
• USD 19 billion on applications and USD 18 billion on infrastructure within that total (Menlo Ventures, 2025).
• Foundation model API market share by usage: Anthropic 40% (up from 24% in 2024), OpenAI 27% (down from 50% in 2023), Google 21% (up from 7% in 2023), others 12% (Menlo Ventures, 2025). If your eval suite is pinned to one provider, that distribution is a portability risk.
• USD 172 billion estimated annual value of generative AI tools to US consumers by early 2026 (Stanford HAI AI Index Report 2026).

What the numbers do not say

Three gaps are worth naming, because their absence is itself informative.

There is no credible public dataset on hallucination rates in production applications, as opposed to benchmark conditions. Every rate on this page comes from a controlled evaluation. Your production rate is almost certainly different and nobody publishes theirs.

There is no public series tracking LLM application uptime, latency regressions, or eval score drift across organisations. Incident counts exist. Reliability metrics do not.

And the eval tooling figures measure whether a team uses evaluation at all, not whether the evaluation is any good. A team with a 20-example smoke test and a team with a 2,000-example graded suite both answer yes.

How to cite this page

genai.qa, “LLM Testing Statistics 2026: Adoption, Hallucination and Eval Benchmarks”, published 6 September 2026, https://genai.qa/blog/llm-testing-statistics-2026/. Figures compiled from the primary sources listed below and last verified 6 September 2026. Where you are citing an individual statistic, please cite the original publisher named alongside it rather than this page.

Methodology and sources

Every statistic on this page was read directly from the source listed below on 6 September 2026. Figures that we could only find in search snippets, secondary summaries, or gated report downloads were excluded rather than repeated. No figure on this page is ours, estimated, or extrapolated.

If you spot a figure here that has been superseded, or a source that has published a newer edition, tell us and we will update the page and move the verification date.