Secure AI Logging for LLM Apps and Agents

LLM applications and AI agents need a security audit trail, but the obvious way to build one, logging every prompt and response, creates a second copy of your most sensitive data in a system that was never designed to hold it. This guide sets out a security logging design for LLM apps and agents: the events to record, how to attribute actions to human and non-human identities, what to keep out of logs, how to protect logs from injection and tampering, and how long to keep them, including the record-keeping articles of the EU AI Act. It is for security architects, detection engineers and platform owners. Tracing, token metrics and quality evaluation are covered in the LLM Observability Guide and are not repeated here.

Key takeaways

Log decisions and actions by default, content by exception. Identity, tool calls, authorisation outcomes and guardrail verdicts belong in the security log; prompt and response text should be off, or held in a separate store with tighter access and shorter retention.
Every agent action needs two names. Record the agent's own identity and the human or service it acted for, otherwise downstream logs show the agent as the actor and the investigation stalls.
AI logs are an attack surface. Model output written to logs can forge entries (CWE-117), and log content read back by AI tools can carry prompt injection, so encode on write and treat log content as untrusted on read.
The never-log list grows with AI. Beyond passwords, tokens and card data, add system prompts, retrieved documents, tool results that echo secrets and raw embeddings.
Retention is a legal decision. The EU AI Act requires logs from high-risk systems to be kept for at least six months, while data protection law limits how long personal data can sit in them; set tiers per field, not per log.

Security logging versus observability

One shared store makes prompts readable by every engineer. Separate the purposes first.

Purpose	Questions it answers	Typical store	Content needed?
Security audit	Who invoked the agent, under which identity, which tools ran, what was allowed or denied	SIEM or security data lake, write-once	Rarely: metadata and hashes
Incident investigation	What exactly did the model see and produce in this session	Restricted content store, referenced from the audit log	Yes, time-limited and access-controlled
Regulatory record-keeping	Traceability of a high-risk AI system over its lifetime	Audit store with defined retention	Depends on the system; see the retention section

The OpenTelemetry GenAI semantic conventions make the same split. Three patterns are listed: the default of not recording instructions, inputs or outputs; recording them on span attributes, which the conventions describe as suited to cases where privacy regulations do not apply or storage complies with them, such as pre-production; and storing content externally with references on the spans, which they recommend for production where sensitive data must be handled securely, because external storage allows separate access controls.

What to log: the AI security event catalogue

OWASP frames every event as when, where, who and what, linked by an interaction identifier: for AI, the conversation ID, with a "who" of at least two parts.

Event	Key fields	Why it matters
Session or conversation start	Conversation ID (`gen_ai.conversation.id`), user identity, client application, agent ID and version, model and provider, prompt template version	Anchors every later event; prompt and model versions explain behaviour changes
Model invocation	Operation name, requested and actual model, response ID, finish reason, token counts, latency	Abuse, Denial of Wallet and key misuse show first as volume and model anomalies
Guardrail or classifier verdict	Which control ran (input filter, output filter, PII detector), verdict, category, action taken (block, redact, allow), confidence	Prompt injection attempts and data leakage are visible here before any harm occurs
Retrieval	Index or data source, number of documents, document IDs or hashes, the identity used for the permission check, sensitivity labels of returned items	Shows what data the model was given and under whose access
Tool or function call	Tool name, call ID, target system, acting identity and on-behalf-of identity, argument hashes or redacted arguments, result status	The point where an agent changes something; the core of agent accountability
Authorisation decision	Policy evaluated, decision (permit or deny), scopes requested and granted, step-up or human approval requested and given	Denied calls and scope elevation are high-value detection signals
Human approval	Approver identity, what was shown to them, decision, time to decide	Proves human oversight happened and who exercised it
Configuration change	System prompt version, tool registry change, model swap, guardrail threshold change, MCP server added or removed, who made it	Silent changes here alter behaviour more than any code release
Credential and token events	Token issued to agent, audience, scopes, lifetime, revocation, secret retrieval by the agent runtime	Links agent behaviour to the non-human identity lifecycle
Logging pipeline health	Exporter failures, dropped spans, redaction processor errors, logging paused	OWASP lists logging start, stop and pause as events in their own right

OWASP warns that a blind checklist creates "alarm fog": start with tool calls, authorisation decisions, guardrail verdicts and configuration changes.

Attribution: recording who really acted

An agent that calls an API with its own credential, on behalf of a user, through an MCP server, produces logs in at least three systems. Unless each hop records both identities and a shared correlation ID, the downstream system sees only the last caller.

Agent identity: record a stable agent identifier. The OpenTelemetry agent conventions define gen_ai.agent.id and suggest the provider-assigned identifier for hosted agents, such as a cloud agent resource ARN.
Principal identity: record the human or upstream service the agent served, taken from the validated token or session, never from a field the model or client can set.
Delegation chain: where agents call agents, log the whole chain. See the Delegation Chains and Attenuated Authority for AI Agents.
No token passthrough: the MCP security best practices explain that when a server forwards a client's upstream token, it cannot tell clients apart and downstream logs show a different identity from the one actually acting. The MCP specification forbids the pattern: servers must not accept tokens that were not issued for them.
Scope elevation: the same MCP guidance tells servers to log elevation events, the scope requested and the subset granted, with correlation IDs.

Each agent also needs an owner, an inventory entry and traceable credentials. See the Agent Identity and Access Guide.

What to keep out of AI logs

Start from the OWASP never-log list (session IDs, access tokens, passwords, connection strings, keys, card data, sensitive personal data, source code). AI systems add categories that arrive inside larger payloads.

Data	Where it hides	Default treatment in the security log
User prompts	Input messages, chat history replayed on every turn	Do not log; record length, language, classifier verdicts and a keyed hash
System prompts	`gen_ai.system_instructions`, debug dumps	Log the template version ID only; OWASP LLM07:2025 System Prompt Leakage applies to your logs
Retrieved documents	Context assembled for retrieval augmented generation	Log document IDs, source and sensitivity label, not text
Tool arguments and results	`gen_ai.tool.call.arguments` and `gen_ai.tool.call.result`	Allowlist safe fields; redact the rest; tool results often echo API keys, records or file contents
Secrets pasted by users	Any free-text field	Scan and mask at the collector before export; alert on detection
Embeddings	Vector payloads in retrieval spans	Never log raw vectors; they can be partially inverted to source text

Redact in the pipeline rather than in each SDK, so that every exporter receives the cleaned record. A fail-closed attribute allowlist beats a pattern blocklist, because new tools add fields without warning. For the legal basis, pseudonymisation and subject rights around security logs generally, see the Privacy in Security Logging and Monitoring Guide.

Protecting AI logs as an attack surface

Injection on write

Model output is attacker-influenced data. If it reaches a text log unencoded, carriage return and line feed characters can forge or split entries (CWE-117, log injection). OWASP recommends structured formats such as JSON, encoding values for the output format, sanitising delimiter characters and limiting the size of user-supplied values. It also notes that parameterised logging alone does not stop CR and LF injection: use a structured encoder that escapes values.

Injection on read

Logs are increasingly read by AI: SOC copilots summarise alerts, and coding agents fetch error reports through MCP servers. Anything an attacker can write into a log can become an instruction to the agent that reads it (MITRE ATLAS AML.T0051 LLM Prompt Injection). The Sentry MCP Agentjacking 2026 research showed a fabricated error report, written with a public write-only key, being returned to coding agents as tool output and acted on. The customer-side lesson is to treat tool output and log content as untrusted input, require approval before agents run commands drawn from external content, and keep long-lived secrets out of agent environments. See the Agentic AI in the SOC Guide.

Integrity and access

Tamper evidence: OWASP calls for tamper detection and moving logs to read-only media as soon as practical. Use write-once storage or hash chaining (AU-9, AU-10).
Least privilege on logs: restrict and periodically review who can read logs, and log access to logs. The restricted content store should be readable only by a named investigation role (AU-9(4)), with every read recorded.
Detect logging stopping: an attacker who controls an agent may try to disable its telemetry. Alert when an agent's event volume drops to zero or redaction processors start failing (ATT&CK T1685 Disable or Modify Tools).

Retention and record-keeping

Regulation (EU) 2024/1689, the EU AI Act, has three logging provisions for high-risk AI systems, as published on the European Commission's AI Act Service Desk (checked November 2026):

Article 12 (record-keeping): high-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system, enabling events relevant to identifying risk situations, post-market monitoring and monitoring of operation by deployers. Annex III point 1(a) biometric systems have a longer minimum list.
Article 19 (providers): providers keep the automatically generated logs under their control for a period appropriate to the intended purpose, of at least six months, unless Union or national law, in particular data protection law, provides otherwise.
Article 26(6) (deployers): deployers keep the logs under their control for at least six months on the same terms; deployers that are financial institutions keep them as part of the documentation required by financial services law.

The application dates of the high-risk obligations were amended by Regulation (EU) 2026/1744 (the Digital Omnibus on AI), in force since 27 July 2026; secondary summaries give 2 December 2027 for Annex III systems and 2 August 2028 for AI in products covered by EU product legislation. Confirm the dates against the official text before planning to them. See the EU AI Act High-Risk AI Systems Guide and the EU AI Act Implementation Timeline Tracker. OWASP's general rule applies either way: do not destroy logs before the required period ends, and do not keep them beyond it.

Tier	Contents	Example retention approach
Audit metadata	Identities, tool calls, decisions, verdicts, configuration changes	Longest: meets regulatory minimums (six months or more for high-risk systems) and your incident investigation window
Restricted content	Prompts, outputs, retrieved text for sessions under investigation or flagged by a guardrail	Short and purpose-bound; extended only by a legal hold on named sessions
Engineering traces	Spans, latency, token metrics without content	Operational need only

How to implement

Inventory AI systems and log paths: list every LLM app, agent, gateway and MCP server, where each sends telemetry today and whether content capture is on.
Define the event schema: adopt the event catalogue above, mapped to OpenTelemetry GenAI attribute names where they exist, and add your own fields for on-behalf-of identity and authorisation decisions.
Turn content off in production: set every instrumentation library to metadata only, then add a restricted content store with references for the use cases that justify it.
Put redaction in the collector: deploy a fail-closed attribute allowlist and secret and personal data patterns at the telemetry collector. Test with seeded canary values (a fake API key, a test card number) and confirm none reach any backend.
Pilot on one agent: check that an investigator can reconstruct a session from metadata plus the content store.
Harden the stores: write-once audit storage, named access for the content store, access logging, and alerts on telemetry gaps.
Roll out and set retention: extend agent by agent, apply the retention tiers with legal sign-off, and keep a rollback path (the previous logging configuration) until detections are validated.

Detection and measurement

Useful detections: repeated authorisation denials for one agent; tool calls outside its normal set; scope elevation followed by sensitive actions; guardrail blocks clustered on one user or document; secrets in prompts; out-of-window configuration changes; and telemetry that stops while the agent's credential is still in use.

Attribution coverage: percentage of tool-call events carrying both agent and principal identity (target 100%).
Content exposure: number of production systems exporting prompt or output text to a general-access backend (target zero).
Canary leakage: seeded canary values found in any log store per test cycle (target zero).
Reconstruction time: time for an investigator to rebuild a flagged agent session end to end.

Framework mapping

Framework	Reference	How it applies
NIST SP 800-53 Rev. 5	AU-2, AU-3, AU-12	Event selection, record content and generation for AI events
NIST SP 800-53 Rev. 5	AU-9, AU-9(4), AU-10, AU-11	Protecting logs, restricting privileged access, non-repudiation, retention
NIST CSF 2.0	PR.PS-04, DE.CM-03	Log records generated and available; activity monitored
OWASP Top 10 for LLM Applications 2025	LLM02, LLM05, LLM07, LLM10	Sensitive data in logs, output handling, system prompt leakage, consumption monitoring
MITRE ATLAS	AML.T0051, AML.T0057	Prompt injection via log content; data leakage through logs

Common failure patterns

Framework defaults left on: an SDK or gateway captures full prompts because nobody changed the default, and the trace backend becomes the largest store of customer data in the company.
Agent shown as the only actor: logs record the agent's service account but not the user it served, so an investigation cannot tell a compromised user from a misbehaving agent.
Redaction in one place only: masking is configured in one SDK, while a second exporter ships the raw spans elsewhere.
AI trusting logs: an agent acts on text an outsider wrote into an error report.
One retention for everything: prompts are kept as long as audit records.

Practitioner checklist

Inventory every AI app, agent, gateway and MCP server and record whether prompt or output content capture is enabled.
Set content capture to off in production instrumentation and use a separate, access-controlled content store with references where content is justified.
Log tool calls, authorisation decisions, guardrail verdicts, human approvals and configuration changes with a conversation ID.
Record both the agent identity and the on-behalf-of principal on every tool call, taken from validated tokens.
Deploy fail-closed attribute allowlists and secret and personal data redaction at the collector, and test with canary values.
Write AI logs in a structured format with an escaping encoder, and treat log and tool output as untrusted when AI reads it.
Store audit logs write-once, restrict and log access to them, and alert when an agent's telemetry stops.
Agree retention tiers with legal, including the six-month minimum for high-risk systems under the EU AI Act where it applies.

Secure AI logging: what to log for LLM apps and agents without leaking data