> For the complete documentation index, see [llms.txt](https://handbook.harmonic.security/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://handbook.harmonic.security/handbook/5.-threats-and-adversarial/5.6-incident-response-for-ai-system.md). # 5.6 Incident response for AI system An AI incident response runbook covering tokens, API keys, MCP grants, apps, connectors, automations, browser permissions, desktop access, and hooks. *Last reviewed: August 18, 2026* {% hint style="info" %} AI incidents need familiar IR discipline plus AI-specific containment: revoke tokens, disable tools, kill tasks, preserve prompts, and review outputs. {% endhint %} ## What security teams need to know about incident response for AI system AI incident response starts with the same fundamentals: scope, contain, preserve evidence, eradicate, recover, and communicate. The difference is the surface area. An AI incident may involve prompts, tool calls, connectors, MCP servers, browser actions, scheduled tasks, artifacts, API keys, and model outputs. The runbook should name each containment lever before the incident. ## Common incident response for AI system security failures * IR teams cannot revoke connector grants quickly. * Logs expire before the investigation begins. * Artifacts or scheduled tasks keep running after account containment. * The team rotates API keys but misses workload identity mappings. * The evidence plan assumes one audit feed covers every authentication path, and the incident used the path it does not cover. * Communications ignore customers or employees whose data appeared in outputs. ## Incident response for AI system security controls checklist * Create containment steps by surface: account, agent identity, connector, MCP server, plugin, hook, scheduled task, memory, knowledge source, artifact, API key, runtime, queue, and endpoint. * Preserve prompts, outputs, tool definitions and results, approvals, files, memory, retrieval sources, versions, spend, and telemetry before cleanup. * Stop active and queued work, then verify downstream actions have stopped. * Revoke or rotate credentials linked to the incident chain and invalidate cached sessions. * Block malicious destinations, tools, skills, packages, retrieval sources, and persistent memory. * Quarantine generated files, code, messages, sites, and other published artifacts. * Preserve a known-good version and define rollback or rebuild criteria. * Notify the provider when the incident involves platform behavior, a vulnerability, or data the provider may hold. * Test legal, privacy, customer, regulator, and law-enforcement notification decision paths. * Run a post-incident control review and add the scenario to red-team and detection coverage. ## Containment order 1. **Stop expansion:** pause tasks, schedules, queues, remote dispatch, publishing, and new sessions. 2. **Cut authority:** revoke agent and user sessions, OAuth grants, API keys, service accounts, and brokered credentials. 3. **Block the path:** disable the affected tool, connector, MCP server, skill, plugin, hook, memory, knowledge source, domain, or model route. 4. **Preserve evidence:** snapshot configuration and versions, export available records, and secure target-system and endpoint logs. 5. **Find outputs:** identify files, code, messages, tickets, transactions, sites, and shared links created by the incident. 6. **Recover deliberately:** restore a known-good configuration, rotate credentials, remove poisoned state, and validate controls before resuming. A user-visible stop button is not proof of containment. Verify that hosted tasks, child agents, browser actions, queued jobs, webhooks, and downstream systems have also stopped. ## Anthropic ### Overview Anthropic AI incident response should include disabling connectors, pausing Cowork and scheduled tasks, revoking OAuth grants or API keys, blocking desktop extensions or MCP servers, disabling Claude Code plugins, hooks, and skills, stopping Managed Agent runs, preserving prompts and tool results, and exporting available telemetry. Know the evidence and containment levers before the incident. Cowork is not currently captured in Anthropic audit logs, the Compliance API, or data exports; Team and Enterprise organizations need Cowork OpenTelemetry configured in advance. Claude Office add-ins are also outside the Compliance API and data exports, with custom Office-agent OpenTelemetry as the enterprise evidence path. Managed Agents keep server-side event history, and managed settings can restrict MCP servers, plugins, hooks, and permission rules organization-wide. For Claude Tag, containment can include removing the app from a channel, disabling DMs, narrowing member access, removing an Access bundle or repository grant, revoking the dedicated service account, and pausing routines. Preserve the Slack thread, channel memory, Audit view, optional Agent Proxy network-event export, and target-system logs. The Audit view does not contain a per-action record of every request and requester, so do not wait until an incident to test correlation. ### Anthropic documentation * [Use connectors to extend Claude's capabilities](https://support.claude.com/en/articles/11176164-use-connectors-to-extend-claude-s-capabilities) * [Claude Managed Agents overview](https://platform.claude.com/docs/en/managed-agents/overview) * [Claude Code settings](https://code.claude.com/docs/en/settings) * [Claude Code authentication](https://code.claude.com/docs/en/team) * [Claude Cowork architecture overview](https://support.claude.com/en/articles/14479288-claude-cowork-architecture-overview) * [Monitor Claude Cowork activity with OpenTelemetry](https://support.claude.com/en/articles/14477985-monitor-claude-cowork-activity-with-opentelemetry) * [Work across Microsoft 365 apps](https://support.claude.com/en/articles/13892150-work-across-microsoft-365-apps) * [Restrict where Claude Tag operates](https://claude.com/docs/claude-tag/admins/restrict-access) * [Review what Claude Tag has done](https://claude.com/docs/claude-tag/admins/audit) ### Applicable Harmonic guides for Anthropic * [Securing Claude Cowork: A Security Practitioner's Guide](https://www.harmonic.security/resources/securing-claude-cowork-a-security-practitioners-guide) ## OpenAI ### Overview OpenAI AI incident response should include revoking Work access where needed, pausing scheduled tasks and Workspace Agents, disabling affected plugins or apps, unpublishing Sites, revoking source-system grants, tightening desktop Work or Codex permission profiles, rotating API keys or workload identity mappings, preserving traces, and pulling governance or compliance exports. The evidence traps are documented. OpenAI says the Compliance API covers user messages and responses across Chat, Work, and Codex, but the Compliance Logs Platform does not capture files, actions, or tool calls. Those details need desktop, connector, target-system, repository, or API evidence. API-key-authenticated Codex usage still follows API organization settings rather than ChatGPT compliance exports. ### OpenAI documentation * [Admin Controls, Security, and Compliance in apps](https://help.openai.com/en/articles/11509118-admin-controls-security-and-compliance-in-apps-enterprise-edu-and-business) * [Compliance APIs for Enterprise Customers](https://help.openai.com/en/articles/9261474-compliance-apis-for-enterprise-customers) * [Codex Governance](https://developers.openai.com/codex/enterprise/governance) * [Agent approvals and security](https://developers.openai.com/codex/agent-approvals-security) * [Workload identity federation](https://developers.openai.com/api/docs/guides/workload-identity-federation) * [ChatGPT Work Admin FAQ](https://learn.chatgpt.com/docs/enterprise/work-admin-faq) * [Plugins in ChatGPT and Codex](https://help.openai.com/en/articles/20001256) * [Creating and managing ChatGPT Sites](https://help.openai.com/en/articles/20001339) ### Applicable Harmonic guides for OpenAI * [Securing ChatGPT Enterprise Guide](https://www.harmonic.security/resources/securing-chatgpt-enterprise-guide) * [Securing Codex Best Practice](https://www.harmonic.security/resources/securing-codex-best-practice) ## Frequently asked questions about incident response for AI system ### What is an AI security incident? It is an event where AI tools, prompts, agents, data, credentials, or outputs create unauthorized access, disclosure, action, or loss of control. That includes chains no human triggered directly, such as an injected agent posting data outward through an approved connector. Treat model outputs that expose someone else's data as incidents too, not just account compromises. ### What should be contained first? Contain active access paths: accounts, API keys, connectors, MCP servers, plugins, hooks, scheduled tasks, and shared artifacts. Automations deserve early attention because they keep running after a user account is contained. Revoke OAuth grants at the source system as well as inside the AI platform. ### What evidence should be preserved? Preserve prompts, outputs, tool calls, approvals, files, connector logs, endpoint logs, and audit exports. Pull compliance exports early, since retention windows may be shorter than the investigation. Record which surfaces have no platform-side evidence so gaps are documented rather than discovered later. ### How are AI incidents different from SaaS incidents? AI incidents often involve generated content and tool chains alongside user clicks or API calls. The actor may be an agent following injected instructions under a legitimate identity, which breaks the assumption that authenticated activity is intended activity. Scoping therefore includes what the model read, not just what the account did. ### What should happen after recovery? Close the loop on the controls that failed: update permission rules, connector scopes, and approval policies to block the chain that worked, and add the incident's technique to the red-team test set so the fix gets retested. Fill any evidence gaps the investigation exposed; an audit feed that did not cover the surface involved is a finding in itself. Then brief affected users on workflow changes, and record what the vendor was told and when. ## Applicable regulations and frameworks | Governance page | Relationship to this article | | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | G.1 Map your controls to NIST AI RMF and CSF 2.0 | This article supplies implementation evidence for the NIST AI RMF and matching NIST CSF 2.0 outcomes. | | G.2 EU AI Act obligations for deployers | Conditional: for an in-scope high-risk system, this supports monitoring, serious-incident escalation, evidence preservation, and reporting. | | G.3 DORA and AI resilience in financial services | Conditional: for a DORA-regulated workflow, this supports resilience testing, vulnerability handling, and incident response. | | G.5 SANS Critical AI Security Guidelines mapping | This article implements relevant SANS Inference Security, Monitoring, and GRC guidance. | | G.6 Write an AI Acceptable Use Policy that holds up | This article supplies a technical or process control used to enforce the acceptable-use policy. | | G.7 Ownership and RACI for AI security | This control depends on the ownership and evidence responsibilities defined in the RACI. | | G.8 ISO/IEC 42001 AI management system | This article supports ISO/IEC 42001 AIMS preparation through incident learning, corrective action, and improvement. | | G.9 HIPAA controls for AI systems handling PHI | Conditional: for a workflow handling ePHI, this supports HIPAA risk analysis, safeguard testing, and security-incident response. | *G.2, G.3, G.4, and G.9 are conditional mappings. They apply only when the deployment is within the legal or regulatory scope described on the linked governance page.* ## Related handbook guidance * [5. Threats & Adversarial](/handbook/5.-threats-and-adversarial.md) * [6.6 Evidence by surface and investigation paths](/handbook/6.-observability-audit-and-evidence/6.6-evidence-by-surface-and-investigation-paths.md) * [6.5 Routing AI telemetry to your SIEM](/handbook/6.-observability-audit-and-evidence/6.5-routing-ai-telemetry-to-your-siem.md) * [5.7 Threat modeling AI systems](/handbook/5.-threats-and-adversarial/5.7-threat-modeling-ai-systems.md) * [G.7 Ownership and RACI for AI security](/reference/governance-and-frameworks/g.7-ownership-and-raci-for-ai-security.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://handbook.harmonic.security/handbook/5.-threats-and-adversarial/5.6-incident-response-for-ai-system.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.