5. Threats & Adversarial

The attack patterns unique to AI systems — prompt injection, tool-based exfiltration, supply-chain compromise, tool poisoning, confused deputy, and AI red-team response.

Last reviewed: August 18, 2026

AI systems introduce attack patterns that have no equivalent in traditional security — prompt injection runs through every other risk in this handbook. Understanding how AI-specific attacks work is a prerequisite for building controls that actually hold.

Articles in this section

1. 5.1 Prompt injection: the connective risk
2. 5.2 Data exfiltration via tools and connectors
3. 5.3 Supply chain attacks and notable CVEs
4. 5.4 Agent-specific threats: tool poisoning and confused deputy
5. 5.5 Red-teaming AI systems
6. 5.6 Incident response for AI system
7. 5.7 Threat modeling AI systems

Frequently asked questions

What is prompt injection and why does it matter more than other AI risks? Prompt injection is crafted input that causes a model to treat attacker-controlled content as instructions. Direct injection comes from the user's prompt. Indirect injection arrives through content the model reads, such as a webpage, document, email, codebase, or tool response. It can steer responses or tool use, but the impact depends on the data and tools available to the model, the privileges granted, and the approvals required. Treat external content as untrusted and constrain resulting actions with deterministic policy.

How do AI agents get used to exfiltrate data? A common path starts with indirect injection, reaches data the agent can access, and then uses a permitted tool or external destination to disclose that data. The connector call may use a legitimate identity, so authentication alone does not stop the attack. Use least-privilege connector scopes, destination and egress controls, confirmation for consequential actions, content inspection, and audit logging. Do not allow untrusted content to drive privileged actions without an independent check.

What makes AI supply chain attacks different from traditional software supply chain attacks? AI supply chain risk includes malicious packages, extensions, MCP servers, skills, hooks, and repository-controlled configuration. In vulnerable or permissive tools, opening, installing, or trusting a project can execute code or influence agent behavior. Not every client executes project content on open, so treat that as a product-specific risk rather than a universal behavior. Review trust prompts, allowlist extensions and MCP servers, scan and pin dependencies, and protect hook and CI configuration.

Which agent threats are easiest to miss? Persistent memory poisoning, spoofed tool results, denial-of-wallet, SSRF or internal network pivoting, cross-workspace boundary failures, data poisoning, and replay or approval race conditions often sit between teams. Include them in the threat model, red-team catalog, detections, and incident runbook rather than assigning them to one control layer.

Related handbook guidance

• AI Security Handbook
• 5.7 Threat modeling AI systems
• 6.6 Evidence by surface and investigation paths
• 7.2 Pilot design and success metrics