EU AI Act Adversarial Testing: Red-Team Checklist

The EU AI Act adversarial testing requirements land for high-risk systems in August 2026, and almost no QA vendor has published a concrete evidence checklist for what “documented adversarial testing” legally means. This post is that checklist. It is vendor-neutral, audit-ready, and built around one idea: you run a single red-team campaign and document it three ways - against EU AI Act Article 15, NIST AI RMF, and the OWASP LLM Top 10 - instead of guessing.

If you want the broader regulatory picture first, read our EU AI Act compliance guide for startups. This page is narrower and more technical: it is the red-team testing checklist for the adversarial-robustness clause specifically.

What Article 15 actually requires for adversarial testing

Article 15 requires high-risk AI systems to be tested for robustness against adversarial inputs and to retain documented evidence of that testing.

That is the whole thing in one sentence. The legal text is broader - it talks about an “appropriate level of accuracy, robustness and cybersecurity” - but the operative phrase for GenAI teams is that high-risk systems must be resilient against attempts to alter their use, behaviour or performance by exploiting vulnerabilities, and resilient against adversarial inputs. You cannot claim resilience you never tested. So Article 15 implies documented red-teaming even though it never says the words.

Who it applies to. Article 15 attaches to high-risk AI systems, and “high-risk” is defined by Annex III. If your GenAI product operates in any of these domains, you are in scope:

• Credit scoring and creditworthiness assessment
• Biometric identification and categorisation
• Critical infrastructure management (water, gas, electricity, traffic)
• Employment - CV screening, candidate ranking, performance evaluation
• Essential services - access to public benefits, emergency services, insurance pricing
• Education - admissions, scoring, proctoring
• Law enforcement and migration / border control

A general-purpose chatbot is usually not high-risk. The same model wired into a loan-approval flow or a hiring funnel is.

The August 2026 timeline. The high-risk obligations apply from August 2026. The phrase that trips people up is “placed on the market.” If you ship a high-risk system after that date - even a system you started building in early 2026 - it must already carry conformity documentation, and the adversarial-testing evidence is part of that documentation. Treating the deadline as “when we start” rather than “when the evidence must already exist” is the most common scheduling mistake.

The red-team evidence Article 15 expects you to keep

Regulators do not grade your intentions; they read your artifacts. There are four documentation primitives that turn a red-team campaign into audit material:

1. Who tested - tester identity and their independence from the build team.
2. What access level - black-box, grey-box, or white-box, because the access level defines how thorough the test could have been.
3. How long and what scope - the date range and the attack surface actually covered.
4. What was found - per-vector results, severity, remediation owner, and the retest outcome.

Miss any one of these and the evidence is weak. A pass/fail summary with no tester identity reads as marketing. A list of attacks with no remediation trail proves you looked but not that you fixed anything.

Why self-attestation fails. A conformity assessment is an evidence exercise. “We tested it and it is robust” is a claim; the four primitives above are the proof. If you cannot show the logged attack vectors, the thresholds you tested against, and the dated sign-off, an assessor has nothing to verify and the attestation does not hold.

Internal QA vs external red team. Internal testing can satisfy Article 15 - if it is genuinely independent and documented to the same standard you would expect from an outside firm. The failure mode is independence: when the people who shipped the model also certify it is safe, the evidence loses credibility. For higher-risk Annex III categories, an external red team is increasingly the expected baseline. Our Series B AI safety questions post covers how investors and auditors probe exactly this independence gap.

Retention. Keep the records as part of the technical documentation file for the lifetime the system is on the market plus the retention period required for high-risk systems (generally ten years after the system is placed on the market). The red-team evidence is not a one-off report you discard after launch - it lives in the documentation file and gets updated when the model or its guardrails change.

One campaign, three frameworks: the crosswalk table

Here is the part that saves you weeks. You do not run three separate test programs for EU AI Act Article 15, NIST AI RMF, and the OWASP LLM Top 10. You run one red-team campaign and map each attack to all three frameworks at once. A single prompt-injection test satisfies OWASP LLM01, the NIST MEASURE function, and the Article 15 robustness clause in one pass.

For the full attack methodology behind the OWASP column, see our OWASP LLM Top 10 testing checklist. The ISO/IEC 42001 column is for teams already pursuing the AI management-system certification - the same red-team evidence feeds your 42001 operational controls without extra work.

The takeaway: one red-team, documented three ways. If a vendor proposes three separate compliance programs, they are billing you for redundant work.

The Article 15 red-team checklist (copy-paste)

Use this as the literal structure of your campaign. Each line maps to the documentation primitives above.

Pre-test (scoping)

• Risk classification confirmed - system is high-risk under Annex III, category named
• Access level agreed and recorded - black-box / grey-box / white-box
• Test scope defined - which endpoints, models, and guardrails are in scope
• Success / fail thresholds set per attack class before testing starts
• Tester independence confirmed and documented

Attack coverage (the vectors)

• Prompt injection - direct and indirect, including injected instructions in retrieved context
• Jailbreak - persona, encoding, and multi-turn escalation attacks
• Data extraction / PII leakage - training-data and system-prompt extraction
• Safety-boundary bypass - eliciting prohibited or unsafe outputs
• Adversarial robustness on inputs - perturbed, obfuscated, and out-of-distribution inputs
• Excessive agency - unsafe tool calls and agent action chains (for agentic systems)
• Unbounded consumption - resource-exhaustion and denial-of-service prompts

Evidence capture (per vector)

• Attack-vector log - the exact payload or technique used
• Pass / fail result against the pre-set threshold
• Severity rating (e.g. critical / high / medium / low)
• Remediation owner and target date
• Retest result after the fix

Sign-off (the audit trail)

• Tester identity and statement of independence
• Date range of the campaign
• Tooling and model versions used
• Pointer to where this evidence sits in the conformity documentation file

That checklist is your Article 15 red-team evidence template - copy it, fill every box, and you have an artifact an assessor can actually verify. If you need help building a high-risk GenAI evidence file from scratch, book a Red-Team Sprint to produce your Article 15 evidence file and we will run this checklist end to end.

Common ways startups fail the documentation test

Most failures are not technical. They are documentation failures that look fine until an assessor reads them.

• The one-time vibe-check. Someone spent an afternoon trying to jailbreak the model, it mostly held, and that became “we tested it.” No logged vectors, no thresholds, no severity, no retest. This is the single most common failure and it is indistinguishable from doing nothing.
• Guardrails without adversarial testing. Plenty of teams ship guardrails - a moderation filter, a system prompt with rules - but never adversarially test whether those guardrails actually hold under attack. Article 15 is about resilience, not configuration. Closing exactly this gap is what red-team sprints exist for.
• No independence trail. The build team tested its own system and signed off. Without a documented independence statement, the evidence reads as self-promotion rather than audit material.
• Leaving it to the deadline. Compliance testing gets pushed to the conformity-assessment date instead of being built into the release process. By then the model has changed, the evidence is stale, and there is no time to remediate findings.

How to produce audit-grade evidence in one sprint

You can build the full evidence artifact in a single focused campaign. A 5-day Red-Team Sprint delivers against the checklist directly:

• 100+ attack vectors run across prompt injection, jailbreak, data extraction, safety bypass, and adversarial robustness
• Every vector scored, logged, and severity-rated with a remediation owner
• The whole campaign mapped to Article 15, NIST AI RMF, and OWASP LLM Top 10 so it satisfies all three at once
• A signed, dated evidence file with the independence trail, ready to drop into your technical documentation

Pair it with a Compliance QA Sprint when you need the full conformity documentation file - not just the adversarial-testing evidence, but the surrounding accuracy, cybersecurity, and risk-management records that complete the Article 15 picture. For agent-based high-risk systems, add an agentic AI safety assessment to cover excessive-agency and tool-use vectors in depth.

On independence: an external red team is not just a thoroughness upgrade, it is an evidence upgrade. When the tester is independent of the build team, the documentation reads as audit material rather than marketing - which is precisely what a conformity assessment is checking for.

Book your Article 15 evidence file

A legal deadline is a hard date, and August 2026 is closer than it looks. If you have a high-risk GenAI system and need defensible, independent, framework-mapped adversarial-testing evidence, this is the work you cannot credibly do in-house.

Book a Red-Team Sprint to produce your Article 15 evidence file - 100+ scored attack vectors, mapped to EU AI Act Article 15, NIST AI RMF, and OWASP LLM Top 10, delivered as a signed conformity-ready evidence artifact. Engagements from AED 15k.

Book a free scope call to confirm your risk classification and the right campaign for your deadline.