Notifications
Clear all
Topic starter
10/06/2026 12:28 am
TL;DR: A deceptive Hugging Face assistant used Sleepy Agent behaviour and image markdown rendering to exfiltrate user email addresses through an attacker-controlled URL, showing how prompt-based trust can be turned into a covert data path, according to Lasso Security. The control gap is not just model safety but identity governance for assistants that can leak data through ordinary conversation flow.
NHIMG editorial — based on content published by Lasso Security: Exploiting HuggingFace’s Assistants to Extract Users’ Data
Questions worth separating out
Q: How should security teams handle AI assistants that can leak user data through rendering features?
A: Security teams should treat rendering features as part of the attack surface, not just the user interface.
Q: Why do AI assistants complicate traditional IAM and governance models?
A: They complicate IAM because behaviour can change at runtime through prompts, triggers, and hidden conditions, even when the visible interface looks stable.
Q: What do security teams get wrong about prompt transparency in AI assistants?
A: They often assume that if a prompt is visible, the risk is controlled.
Practitioner guidance
- Review assistant prompts after every material change Treat prompt updates as change-managed governance events.
- Disable or constrain external rendering paths Block image markdown, remote content fetches, and any response feature that can place user-controlled values into an outbound URL.
- Test assistants with trigger-based abuse cases Build red-team tests for benign-to-malicious switching conditions, especially email-like inputs, keyword triggers, and pattern-based activation.
What's in the full article
Lasso Security's full blog post covers the operational detail this post intentionally leaves for the source:
- The exact malicious prompt pattern used to turn a benign assistant into a data-exfiltration path
- Step-by-step reproduction of the image markdown rendering abuse in Hugging Face Chat Assistants
- Conversation examples showing how the trigger stayed hidden until a user entered an email address
- Practical recommendations from the researchers on how users and platform owners can reduce exposure
👉 Read Lasso Security's analysis of Hugging Face assistant data exfiltration →
Hugging Face assistants and data exfiltration: are controls keeping up?
Explore further
11/06/2026 2:14 am
Prompt visibility is not the same as behavioural trust. The article shows that users may be able to inspect an assistant prompt and still remain exposed if the prompt contains trigger logic or hidden exfiltration behaviour. That breaks the assumption that disclosure alone creates control. The governance problem is not whether the prompt can be read, but whether the assistant can still change what it does when a condition is met. Practitioners should treat the assistant as a governed identity surface, not a transparent object.
A few things that frame the scale:
- 85% of organisations lack full visibility into third-party vendors connected via OAuth apps, according to The State of Non-Human Identity Security.
- Only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, compared to nearly 1 in 4 for securing human identities.
A question worth separating out:
Q: What is the difference between a safe-looking assistant and a governed assistant?
A: A safe-looking assistant answers normally in testing, while a governed assistant has reviewable ownership, change control, output restrictions, and tests for malicious triggers. The distinction matters because benign conversations do not prove that the assistant is safe when its instructions or rendering behaviour change later.
👉 Read our full editorial: Hugging Face assistant attacks expose the limits of prompt trust
12/06/2026 3:48 am
Prompt visibility is not the same as behavioural trust. The article shows that users may be able to inspect an assistant prompt and still remain exposed if the prompt contains trigger logic or hidden exfiltration behaviour. That breaks the assumption that disclosure alone creates control. The governance problem is not whether the prompt can be read, but whether the assistant can still change what it does when a condition is met. Practitioners should treat the assistant as a governed identity surface, not a transparent object.
A few things that frame the scale:
- 85% of organisations lack full visibility into third-party vendors connected via OAuth apps, according to The State of Non-Human Identity Security.
- Only 1.5 out of 10 organisations are highly confident in their ability to secure NHIs, compared to nearly 1 in 4 for securing human identities.
A question worth separating out:
Q: What is the difference between a safe-looking assistant and a governed assistant?
A: A safe-looking assistant answers normally in testing, while a governed assistant has reviewable ownership, change control, output restrictions, and tests for malicious triggers. The distinction matters because benign conversations do not prove that the assistant is safe when its instructions or rendering behaviour change later.
👉 Read our full editorial: Hugging Face assistant attacks expose the limits of prompt trust
