6. Observability, Audit & Evidence

What security teams can and cannot see across AI platforms — compliance APIs, usage logs, OpenTelemetry, SIEM routing, investigation paths, and review cadence.

Last reviewed: August 18, 2026

You cannot govern what you cannot see. Every AI platform has audit blind spots — knowing exactly where each platform's coverage stops is what separates a real control from a false assumption. This section covers what you can see, how to route it, and how to use it.

Articles in this section

1. 6.1 The audit gap: what you can and can't see
2. 6.2 OpenTelemetry for AI runtime visibility
3. 6.3 Compliance APIs by platform
4. 6.4 Analytics and usage APIs
5. 6.5 Routing AI telemetry to your SIEM
6. 6.6 Evidence by surface and investigation paths
7. 6.7 Continuous review cadence

Frequently asked questions

What are the biggest audit gaps across enterprise AI platforms? The largest gap is assuming one vendor API covers every surface. Coverage varies by product and changes over time. Core chat and administrative events may be available while local agent runs, browser or computer actions, tool arguments, desktop bridges, background tasks, or cross-app data flows are absent or exposed separately. Maintain a per-surface evidence matrix, generate a representative test event on each surface, and supplement vendor records with identity, endpoint, connector, repository, network, and runtime telemetry.

What is OpenTelemetry and why does it matter for AI security? OpenTelemetry is a vendor-neutral framework for traces, metrics, and logs. An instrumented AI application or agent can emit model, agent, and tool spans along with latency, token usage, errors, and selected attributes. It does not automatically create visibility into a closed SaaS product, and prompt, output, and tool-argument content is opt-in or product-specific. Because those fields can contain secrets or personal data, filter or redact them and restrict telemetry access and retention before routing events to a SIEM.

Which compliance API gives me the most coverage for an AI security incident investigation? No single compliance API provides complete incident coverage. Coverage differs by event type, content depth, retention, license, authentication path, and product surface. Test representative actions on every enabled surface, reconcile the resulting provider, identity, endpoint, runtime, network, repository, and target-system records, and document any investigation question the evidence cannot answer.

How do we know an AI telemetry pipeline is reliable? Define expected events, delivery latency, completeness, schema version, timestamp requirements, retention, and reconciliation checks. Generate known test activity and compare the provider or runtime source count with the collector and SIEM count. Alert on silent source stoppage, parsing failures, clock drift, and unexplained sampling.

Related handbook guidance

• AI Security Handbook
• 5.6 Incident response for AI system
• 6.7 Continuous review cadence
• 7.4 The vendor-neutral control matrix