Public conversations understate enterprise risk because users change behaviour when they know others may see the transcript. They ask safer questions, reveal less sensitive context, and avoid obvious misuse. Private enterprise workflows remove that social pressure, so the true control test is how the system behaves when visibility is absent.
Why This Matters for Security Teams
Public LLM conversations create a false sense of safety. People naturally sanitize prompts when they know transcripts may be visible, but enterprise workflows remove that social friction and expose the real control surface: sensitive context, proprietary data, and unsafe tool use. That matters because risk is rarely the question asked in public, but the question asked privately with access attached. Guidance from the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 both point to the same operational issue: the model is only part of the risk, while the surrounding data, permissions, and instructions determine impact.
This is where NHIMG research becomes practical. In the AI Agents: The New Attack Surface report, NHIMG highlights that only 52% of companies can track and audit the data their AI agents access, leaving the rest with a visibility gap that can hide misuse, leakage, and policy drift. In practice, many security teams encounter enterprise AI risk only after an incident review, rather than through intentional testing of private prompts, hidden context, and delegated actions.
How It Works in Practice
The gap between public and enterprise risk comes from environment, not model behaviour alone. Public demos typically omit secrets, internal documents, customer records, and tool permissions. Enterprise deployments do the opposite: they connect the LLM to email, ticketing, code repositories, knowledge bases, and sometimes autonomous actions. Once those integrations exist, prompt content becomes only one signal among many, and the bigger risk is what the system can retrieve, infer, or execute.
Security teams should evaluate the full request path: user input, system instructions, retrieved context, model output, and downstream tool calls. That means testing for prompt injection, data exfiltration, unsafe summarisation, and over-broad agent permissions. The MITRE ATLAS adversarial AI threat matrix is useful for mapping attack techniques, while the NIST AI 600-1 Generative AI Profile helps translate those threats into governance, measurement, and monitoring.
- Test private prompts with realistic internal context, not polished demo inputs.
- Review retrieval filters so the model cannot surface unrelated sensitive records.
- Restrict tools and actions so the model cannot turn a bad answer into a bad transaction.
- Log prompts, retrieved sources, and tool calls for audit and incident response.
NHIMG’s OWASP NHI Top 10 reinforces that identity, token scope, and credential handling become part of the LLM attack surface as soon as the system acts on behalf of a user or service. These controls tend to break down when LLMs are embedded in fast-moving workflows that reuse broad service credentials and unreviewed retrieval sources because the system inherits trust it cannot validate.
Common Variations and Edge Cases
Tighter visibility controls often increase friction, latency, and review overhead, requiring organisations to balance usability against assurance. That tradeoff is especially sharp in customer support, software engineering, and research workflows where users need speed but also handle confidential data. Best practice is evolving, and there is no universal standard for how much transcript visibility, prompt logging, or human review is appropriate across all use cases.
One common edge case is when public and private behaviour diverge in the same platform. A model may appear safe in a shared demo but become risky once connected to internal search, documents, or delegated agents. Another is role confusion: a chatbot that only answers questions in public may become an operator in enterprise, changing the impact from misinformation to unauthorised action. NHIMG’s AI LLM hijack breach research shows how quickly exposed credentials and weak identity controls can turn AI access into an attacker entry point, which is why enterprise AI risk must be reviewed alongside NHI governance.
Public conversations also understate risk in regulated environments where data retention, privacy, and access review matter as much as model quality. That is why practitioners should align AI testing with the NIST Cybersecurity Framework 2.0 and keep the CSA MAESTRO agentic AI threat modeling framework in view when agents can take actions, not just generate text.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATLAS address the attack and risk surface, while NIST AI RMF, NIST AI 600-1 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | AI risk governance is central when private prompts expose enterprise context. | |
| OWASP Agentic AI Top 10 | Agentic app risks include prompt injection, tool abuse, and data leakage. | |
| MITRE ATLAS | ATLAS helps map adversarial AI techniques against private enterprise workflows. | |
| NIST AI 600-1 | The GenAI profile translates model risk into operational controls and monitoring. | |
| NIST CSF 2.0 | PR.AC-4 | Access governance matters when LLMs inherit enterprise permissions and data access. |
Limit AI system privileges and review entitlements as if the model were a high-risk service account.
Related resources from NHI Mgmt Group
- Why does agentic AI create mission drift risk in enterprise environments?
- Why does shadow AI increase enterprise risk even when users are authenticated?
- How can organisations reduce risk from shadow AI agents already inside the enterprise?
- Why do proxy-based controls miss part of enterprise AI risk?
