Lenovo Lena chatbot shows how prompt injection becomes account theft

At a glance

What this is: This is an analysis of a prompt-injection attack against Lenovo’s Lena AI chatbot that turned a product query into session-cookie theft and potential internal access.

Why it matters: It matters because AI chatbots can sit inside identity and support workflows, so weak output controls, over-permissive permissions, and poor session handling can expose human and system identities alike.

👉 Read Swarmnetics' analysis of Lenovo's Lena chatbot prompt-injection attack

Context

Prompt injection is a control failure, not just a model quirk. When a chatbot can be persuaded to treat attacker-controlled text as instruction, the real issue is the absence of hard boundaries between user input, model output, and downstream tooling. In this case, the primary identity risk is session abuse, because stolen cookies can become a shortcut into authenticated workflows.

For IAM and security teams, the governance question is how AI-enabled support systems inherit trust. A chatbot that can reach product information, format responses, and hand off to human agents is operating across multiple trust domains, which makes it relevant to human identity, session management, and privileged workflow design. That starting position is increasingly typical, not exceptional.

Key questions

Q: What breaks when an AI chatbot can treat untrusted text as an instruction?

A: The main failure is that the chatbot stops being a responder and starts acting like a control surface. When output can be reused as input, attackers can redirect formatting, external links, or tool calls. That turns a conversation into workflow manipulation and can expose sessions, accounts, or internal actions.

Q: Why do AI chatbots create session theft risk in customer support flows?

A: Because support flows often combine authenticated browsers, cookies, and human escalation inside one conversation. If the chatbot can influence page content or external requests, bearer tokens may be exposed. The risk grows when the same session context moves from bot to human without revalidation.

Q: What do security teams get wrong about prompt injection in chatbots?

A: They often treat it as a prompt-writing problem when it is really a trust-boundary problem. The issue is not only whether the model resists manipulation, but whether the surrounding application accepts untrusted output, fetches external content, or gives the model tool permissions it should not have.

Q: Who is accountable when an AI assistant overshares sensitive content?

A: Accountability sits with the team that owns the policy, the attribute feeds, and the enforcement points, because ABAC only works when all three are managed together. If any one of them is missing, the organisation has not built a defensible control path, even if the model itself appears constrained.

Technical breakdown

How prompt injection turns chatbot output into attacker-controlled input

Prompt injection works when a model or agent treats untrusted text as if it were a legitimate instruction. In this case, the chatbot could be influenced to emit HTML, JSON, and plain text in a chosen order, then reuse that output as a new instruction path. That creates a loop where attacker-shaped content moves from answer generation into control flow. The security issue is not the language model alone, but the orchestration layer that accepts model output as trusted context. Practical implication: separate user content, model output, and execution logic so output can never become executable instruction.

Practical implication: enforce strict input and output segregation so model output cannot re-enter the control path.

Why session cookies remain a high-value target in AI support workflows

Session cookies are bearer tokens, so whoever holds them can often impersonate the authenticated user until the session expires or is revoked. When an AI chatbot can steer a browser or support interaction toward a remote server, cookie leakage becomes an identity compromise rather than a pure web flaw. The risk increases when human support agents are brought into the same conversation, because agent sessions may inherit broader internal access than the original end user. Practical implication: treat cookies, support escalations, and chat handoffs as identity-sensitive events, not just application events.

Practical implication: bind session handling and escalation paths to explicit identity controls and revocation events.

Why AI chatbots resemble XSS risk patterns, but with broader blast radius

The comparison to cross-site scripting is useful because both abuses rely on untrusted content being interpreted by a trusted system. The difference is that AI chatbots can interact with tools, present different output formats, and influence human decisions, so the attack surface extends beyond the browser. That means sanitisation alone is not enough if the surrounding workflow lets the model call URLs, fetch external content, or route to support staff. Practical implication: review AI assistants as workflow orchestrators and not just as text generators.

Practical implication: assess the full tool chain around the chatbot, including browser actions, external fetches, and human escalation.

Threat narrative

Attacker objective: The attacker’s objective is to steal authenticated session material and use it to reach internal systems or privileged workflows.

1. Entry occurred when the attacker asked the chatbot for information about a specific product and used that conversation as the initial foothold.
2. Escalation happened when the chatbot was induced to emit attacker-shaped output that could be fed back as instruction, allowing control over response structure and remote-link handling.
3. Impact followed when session cookies were directed to a malicious server and a human support interaction created a path toward internal access.

Breaches seen in the wild

• MITRE ATT&CK Enterprise Matrix — MITRE ATT&CK Enterprise — adversary tactics and techniques, threat detection, attack chain mapping, credential access, lateral movement, privilege escalation.
• Meta AI Instagram Account Takeover — 20,225 Instagram accounts hijacked via compromised Meta AI support chatbot with overprivileged access.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Prompt injection is now an identity problem because chatbot trust can be converted into session theft. The important failure is not that the model was fooled, but that the workflow allowed a conversational response to interact with cookie-bearing browser sessions and support escalation. That makes the identity boundary porous, especially where AI assistants sit inside authenticated customer journeys. Practitioners should treat model orchestration as part of the access-control plane.

Chatbots that can format output and call external resources need the same scrutiny long applied to untrusted web input. The article shows a familiar pattern with a new wrapper: if one system can turn output into a link, a fetch, or a follow-on instruction, the model is no longer just generating text. That is why the control gap here is not model intelligence, but output handling and tool permission design. Security teams should review AI assistants under OWASP NHI Top 10 thinking where delegated actions and token exposure intersect.

Human support handoffs are a governance weak point when AI systems sit in front of them. The moment a chatbot can escalate to a human agent, the attack surface expands from model abuse to identity impersonation and privilege inheritance. That is a familiar failure mode in IAM programmes: trust is transferred without re-authentication or step-up checks. Teams should require explicit identity revalidation before a chatbot-to-human transfer can inherit session context.

AI productivity pressure is encouraging organisations to deploy trust-dependent systems before they can observe them properly. Rapid adoption creates the same kind of oversight gap that early web teams faced with script injection, except the blast radius now includes user identity, support tooling, and downstream automation. The named concept here is delegated trust contamination, where one untrusted interaction contaminates the next trust decision in the chain. Practitioners should inventory AI-assisted workflows as potential identity pathways, not standalone features.

From our research:

• The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to The State of Secrets in AppSec.
• 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, according to The State of Secrets in AppSec.
• For a broader breach pattern view, 52 NHI Breaches Analysis shows how identity exposure often turns into downstream access abuse.

What this signals

Prompt-injection testing should now be part of application security review wherever an AI assistant can touch authenticated workflows. The control question is no longer whether the model is clever enough to answer, but whether the surrounding system can prevent output from becoming instruction. That is where identity and access teams need to work with application security.

Delegated trust contamination: once an AI assistant can transfer context into a human support handoff, the original trust decision may be inherited without re-authentication. That creates a practical gap between identity assurance and workflow automation, and it is the kind of boundary error that attackers repeatedly exploit.

Programmes that already track secret exposure, session lifecycle, and support escalation are better positioned to absorb this class of risk. Where those controls are fragmented, even a simple prompt injection can become a multi-system identity issue rather than a contained chatbot defect.

For practitioners

• Separate model output from executable workflow logic Do not let chatbot output flow directly into HTML rendering, URL construction, browser navigation, or tool invocation without strict validation and allowlisting. Treat every output channel as untrusted until it is explicitly normalised and checked.
• Protect session cookies as high-risk credentials Apply HttpOnly, Secure, SameSite, short lifetimes, and rapid revocation for sessions that appear in AI-assisted support journeys. If a chatbot can influence a browser or support agent workflow, session theft becomes an access-control issue.
• Require step-up verification before support handoff When an AI assistant escalates to a human agent, re-authenticate the user and do not inherit the chatbot context as proof of identity. The handoff should create a fresh trust decision, not extend the previous one.
• Test prompt injection as a workflow attack, not a model bug Red-team chatbots with indirect prompt injection, response-format manipulation, and external-link steering. Include browser effects, tool calls, and support workflows in the test scope so the assessment matches the real exposure.

Key takeaways

• The breach pattern is not about model intelligence, but about broken trust boundaries between chatbot output, browser sessions, and human escalation.
• Session cookies and support handoffs turn a prompt-injection path into an identity compromise with real access implications.
• Teams need to test AI assistants as workflow orchestrators and apply identity controls before output can become instruction.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads — causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Persistent Cookie: A persistent cookie is a session cookie that remains valid after the browser closes until it expires or is explicitly removed. In remote access designs, it can improve usability, but it also increases the chance that a reused session becomes standing access if the organisation does not enforce compensating controls.
• Identity Handoff: The controlled transfer of access from one user to the next on a shared device or application session. In manufacturing, the handoff must close the prior session, preserve auditability, and prevent residual access from carrying into the next operator’s activity.
• Delegated Trust Contamination: Delegated trust contamination is a failure mode where one untrusted interaction influences the next trust decision in a workflow. In AI systems, it happens when model output, browser actions, and human escalation are chained without strong separation, letting attacker influence spread across systems.

What's in the full analysis

Swarmnetics' full article covers the operational detail this post intentionally leaves for the source:

• The step-by-step prompt sequence used to steer the chatbot into unsafe output handling and cookie exposure.
• The exact HTML, JSON, and plain-text response pattern that enabled the feedback loop into attacker-controlled instructions.
• The researchers' observations on how the human support handoff expanded the access path beyond the chatbot itself.
• The broader comparison to older injection patterns and why this technique may generalise to other ChatGPT-based deployments.

👉 Swarmnetics' full article covers the attack flow, session-cookie theft path, and support handoff details

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through the NHI Foundation Level course, the industry's only accredited NHI security programme. It is designed for practitioners who need a practical foundation for governing identities, sessions, and delegated access across modern programmes.