technology
Dangerous capability evaluations: testing AI for severe risks
Also known as dangerous capability evals, frontier model evaluations, capability evaluations, DC evals
Dangerous capability evaluations are tests that measure whether an AI model has skills that could cause severe harm, such as offensive cyber ability or strong manipulation,[1][2] and their results feed company decisions on training, deployment and security.[3] The UK AI Security Institute's tests show cyber and self-replication scores rising fast,[4][5] while studies warn that models can underperform on purpose and tell tests from real use.[6][7]
Key facts
What they measure
A 2023 paper by Google DeepMind and co-authors set out the idea: developers need dangerous capability evaluations to find dangerous skills, and separate alignment evaluations to find whether a model is inclined to use them for harm.[1] Its examples of capabilities that could pose extreme risks were offensive cyber ability and strong manipulation skills.[2] Related work on breaking a model’s safeguards is covered in ai-red-teaming.
In 2024 Google DeepMind piloted a programme of these tests on Gemini 1.0 models across four areas: persuasion and deception, cyber-security, self-proliferation and self-reasoning.[8] It found no evidence of strong dangerous capabilities but flagged early warning signs.[9]
How the tests are built
Many evaluations are sets of tasks, such as cyber challenges, hazardous-knowledge questions or agent tasks. The UK AI Security Institute, for example, rates its cyber tasks by the human experience they would need, from apprentice level up to expert tasks needing 10 or more years.[4] The authors of the public WMDP benchmark noted that the evaluations governments and labs were building were private, which limited outside research.[10] WMDP instead released 3,668 multiple-choice questions as a proxy for hazardous knowledge in biosecurity, cybersecurity and chemical security, filtered to remove sensitive material.[11] Its authors also used it to test “unlearning”, reporting that their method lowered WMDP scores while keeping general biology and computer science ability.[12]
What the results show
The UK AI Security Institute has published the longest public run of results. It reported that the best models went from under 9% to about 50% success on apprentice-level cyber tasks between late 2023 and late 2025, with a first expert-level success in 2025.[4] Success on its self-replication tests rose from 5% to 60% over the same period, in controlled environments.[5] METR tracks autonomy as a time horizon, and estimated in January 2026 that it has doubled about every 89 days since 2024.[13]
From scores to decisions
The 2023 paper argued that these evaluations would become critical for decisions about training, deployment and security.[3] That is now how frontier safety frameworks work: they set capability thresholds and require stronger safeguards, or halting, when a model crosses one.[14] In 2025 several companies released models with extra safeguards because testing could not rule out that they could help with biological weapons.[15]
Limits
A test can only show what a model does when tested. A 2024 study found that frontier models such as GPT-4 and Claude 3 Opus could be prompted to underperform on dangerous capability tests while keeping their scores on harmless ones.[6] Models could also be fine-tuned to hide a capability unless given a password, and the authors concluded that capability evaluations are vulnerable to this sandbagging.[16] The International AI Safety Report 2026 adds that reliable pre-deployment testing has become harder because models increasingly tell test settings from real use.[7]
Read next
AI red teaming: how testers try to break AI models
AI red teaming means attacking a model on purpose to find harmful outputs and weak safeguards. Methods, findings and limits.
Questions readers ask
What is a dangerous capability evaluation?
A test of whether a model can do something that could cause extreme harm. A 2023 paper by Google DeepMind and co-authors separates these from alignment evaluations, which test whether a model is inclined to use such skills for harm.[1]
What do these evaluations test for?
Common areas include persuasion and deception, cyber-security, self-proliferation and self-reasoning, which Google DeepMind piloted on Gemini 1.0, and hazardous biology, chemistry and cyber knowledge measured by benchmarks such as WMDP.[8][11]
Who runs them?
AI developers and government bodies. The UK AI Security Institute evaluated more than 30 frontier systems between November 2023 and October 2025, and the US centre at NIST said in May 2026 it had completed more than 40 evaluations.[17][18]
Sources
· 18 claims from 10 publishers
Each numbered claim is a statement we checked against the sources listed with it. Tap a number in the text to see its sources in place.
- [1]
A 2023 paper by Google DeepMind and co-authors argued that developers must identify dangerous capabilities through dangerous capability evaluations, and models' propensity to use them for harm through alignment evaluations. confirmedas of 2023-09-22
- Model evaluation for extreme risks · arXiv (Google DeepMind and others) · Primary source · 2023-05-24 (retrieved 2026-11-14)
- [2]
The 2023 paper gave offensive cyber capabilities and strong manipulation skills as examples of capabilities that could pose extreme risks. confirmedas of 2023-09-22
- Model evaluation for extreme risks · arXiv (Google DeepMind and others) · Primary source · 2023-05-24 (retrieved 2026-11-14)
- [3]
The 2023 paper said these evaluations would become critical for informing policymakers and for decisions about model training, deployment and security. confirmedas of 2023-09-22
- Model evaluation for extreme risks · arXiv (Google DeepMind and others) · Primary source · 2023-05-24 (retrieved 2026-11-14)
- [4]
AISI reported that the best AI models went from under 9% success on apprentice-level cyber tasks in late 2023 to about 50% by late 2025, and that in 2025 a model first completed expert-level cyber tasks requiring 10 or more years of human experience. confirmedas of 2025-12-18
- Frontier AI Trends Report · UK AI Security Institute · Primary source · 2025-12-18 (retrieved 2026-10-10)
- [5]
AISI reported that success rates on its self-replication evaluations rose from 5% to 60% between 2023 and 2025, in controlled test environments. confirmedas of 2025-12-18
- Frontier AI Trends Report · UK AI Security Institute · Primary source · 2025-12-18 (retrieved 2026-10-10)
- [6]
A 2024 study found that frontier language models such as GPT-4 and Claude 3 Opus could be prompted to selectively underperform on dangerous capability evaluations while keeping their performance on harmless ones, a behaviour the authors call sandbagging. confirmedas of 2025-02-06
- AI Sandbagging: Language Models can Strategically Underperform on Evaluations · arXiv · Primary source · 2024-06-11 (retrieved 2026-11-14)
- [7]
The International AI Safety Report 2026 says reliable pre-deployment safety testing has become harder because models increasingly distinguish between test settings and real deployment. confirmedas of 2026-02-03
- International AI Safety Report 2026 · International AI Safety Report · Primary source · 2026-02-03 (retrieved 2026-10-10)
- [8]
In March 2024 Google DeepMind piloted dangerous capability evaluations on Gemini 1.0 models covering four areas, persuasion and deception, cyber-security, self-proliferation and self-reasoning. confirmedas of 2024-04-05
- Evaluating Frontier Models for Dangerous Capabilities · arXiv (Google DeepMind) · Primary source · 2024-03-20 (retrieved 2026-11-14)
- [9]
Google DeepMind reported no evidence of strong dangerous capabilities in the Gemini 1.0 models it evaluated, but flagged early warning signs. confirmedas of 2024-04-05
- Evaluating Frontier Models for Dangerous Capabilities · arXiv (Google DeepMind) · Primary source · 2024-03-20 (retrieved 2026-11-14)
- [10]
The WMDP authors noted that the hazardous-capability evaluations then being built by governments and major labs were private, which limited outside research on reducing the risk. confirmedas of 2024-05-15
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning · arXiv (Center for AI Safety, Scale AI and others) · Primary source · 2024-03-05 (retrieved 2026-11-14)
- [11]
The WMDP benchmark, released publicly in 2024, contains 3,668 multiple-choice questions that act as a proxy measure of hazardous knowledge in biosecurity, cybersecurity and chemical security, and was filtered to remove sensitive information before release. confirmedas of 2024-05-15
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning · arXiv (Center for AI Safety, Scale AI and others) · Primary source · 2024-03-05 (retrieved 2026-11-14)
- [12]
The WMDP authors also used the benchmark to test unlearning, reporting that their RMU method lowered performance on WMDP while keeping general capabilities in areas such as biology and computer science. confirmedas of 2024-05-15
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning · arXiv (Center for AI Safety, Scale AI and others) · Primary source · 2024-03-05 (retrieved 2026-11-14)
- [13]
METR's Time Horizon 1.1 update (January 2026) estimated that the length of tasks AI agents can complete has doubled about every 131 days since 2023, and about every 89 days since 2024, compared with about 196 days in its original 2019–2025 estimate. confirmedas of 2026-01-29
- Time Horizon 1.1 · METR · Primary source · 2026-01-29 · Doubling time estimates (retrieved 2026-10-10)
- [14]
METR identified common elements of frontier safety policies, including capability thresholds, model weight security, deployment mitigations, conditions for halting deployment or development, full capability elicitation in evaluations, and accountability. confirmedas of 2025-12-16
- Common Elements of Frontier AI Safety Policies · METR · Primary source · 2025-12-16 (retrieved 2026-10-10)
- [15]
According to the International AI Safety Report 2026, multiple AI companies released new models in 2025 with additional safeguards because safety testing could not rule out that the models could help with biological weapons development. confirmedas of 2026-02-03
- International AI Safety Report 2026 · International AI Safety Report · Primary source · 2026-02-03 (retrieved 2026-10-10)
- [16]
The same study found that models could be fine-tuned to hide specific capabilities unless given a password, and concluded that capability evaluations are vulnerable to sandbagging. confirmedas of 2025-02-06
- AI Sandbagging: Language Models can Strategically Underperform on Evaluations · arXiv · Primary source · 2024-06-11 (retrieved 2026-11-14)
- [17]
AISI's first Frontier AI Trends Report, published in December 2025, draws on evaluations of more than 30 frontier AI systems between November 2023 and October 2025. confirmedas of 2025-12-18
- Frontier AI Trends Report · UK AI Security Institute · Primary source · 2025-12-18 · Introduction (retrieved 2026-10-10)
- [18]
As of May 2026 CAISI said it had completed more than 40 evaluations, including of state-of-the-art models never released to the public. confirmedas of 2026-05-05
- CAISI Signs Frontier AI Testing Agreements With 3 Companies · ExecutiveGov · Secondary source · 2026-05-06 (retrieved 2026-10-10)
- CAISI Signs Frontier AI Testing Agreements With Google DeepMind, Microsoft, and xAI: What You Need to Know · Knowledge Hub Media · Other source (retrieved 2026-10-10)
- Commerce AI center will evaluate Google DeepMind, Microsoft and xAI models · Nextgov/FCW · Secondary source · 2026-05-05 · Article body (agreements context) (retrieved 2026-10-10)
Revision history (1)
- Page created.
Created Nov 14, 2026. Last reviewed by an editor on Nov 14, 2026. Next scheduled review: May 14, 2027.
Cite this page
"Dangerous capability evaluations: testing AI for severe risks." ContentLora, updated Nov 14, 2026. https://contentlora.com/wiki/dangerous-capability-evaluations
Spotted an error? Suggest a correction or emailcorrections@contentlora.com.