The biggest failure mode is unsanctioned use through personal or unmanaged accounts, because security teams lose visibility, auditability, and enforcement. The risk is not limited to model output. It includes whatever the employee enters, how the service handles it, and whether the organisation can prove policy was applied.
Why unmanaged ChatGPT use is the real failure mode
The biggest failure mode is not simply that an employee gets a wrong answer. It is that work gets pushed into a consumer or personal account outside the organisation’s control, so the organisation loses the ability to see, govern, and prove what happened. That turns a productivity tool into an unmanaged data path, and the exposure is driven by the account and policy boundary as much as by the model output itself.
Once a worker uses a personal or otherwise unmanaged account, the organisation may no longer control retention, logging, access review, or deletion. The practical question is not whether ChatGPT was used, but whether the use happened through an approved channel that the business can actually govern.
That is why the failure mode matters even when the prompt seems harmless. The employee may paste source code, customer data, internal plans, or incident details into a service that is outside corporate visibility, and the organisation may have no reliable way to reconstruct what was submitted, where it was stored, or who could later access it.
What makes the risk larger than a bad AI answer
The dangerous part is the loss of control over the full interaction lifecycle. If an employee can use an unsanctioned account, security cannot consistently enforce acceptable use, data handling rules, retention settings, or offboarding controls. That makes the account boundary, not the model quality, the weakest point in the workflow.
This is also where shadow AI becomes a governance issue. Samsung ChatGPT leak 2023 shows how quickly routine employee use can create data leakage when sensitive material is entered into an unmanaged generative AI service. The lesson is less about one company than about how easily internal information escapes normal control paths.
Even when the organisation later approves the tool, earlier unsanctioned use may already have created an exposure window. Employees often do not distinguish between “I was experimenting” and “I disclosed regulated or confidential material,” but security teams must, because those cases differ in auditability, containment, and legal defensibility.
How to judge whether ChatGPT use is controlled or merely tolerated
A controlled deployment gives the organisation a way to answer four questions: who used it, under what policy, what data was allowed, and what records exist. If those answers are unclear, the use is effectively unmanaged even if leadership has verbally “allowed AI.”
Look first at the account path. Approved enterprise access, identity-linked logging, and centrally managed policy enforcement are materially different from personal sign-up, shared credentials, or browser-based experimentation. The strongest signal of risk is when the same employee can move between sanctioned and unsanctioned use without detection.
Then look at data handling. If employees are likely to paste code, secrets, client material, or drafts containing confidential context, the organisation needs explicit rules and technical controls around what may be entered. Where that control is absent, the model becomes a persistence point for business information that the organisation may not be able to retrieve or constrain later.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.OC-01 — Organizational Context | Workplace ChatGPT use must fit the org's approved context and data handling rules. |
| Recommendation — Define approved AI-use boundaries and make employees use only sanctioned channels. | ||
| NIST SP 800-53 Rev 5 | AU-2 — Event Logging | Unmanaged accounts remove the audit trail needed to reconstruct use and data disclosure. |
| AC-6 — Least Privilege | Employees should only have the AI access and data exposure needed for their role. | |
| Recommendation — Log AI access and prompt activity through managed enterprise controls. Restrict AI access and data-sharing permissions to the minimum necessary. | ||
| ISO/IEC 27001:2022 | A.5.10 — Acceptable use of information and associated assets | Employee ChatGPT use needs clear acceptable-use rules for what may be entered and where. |
| A.5.15 — Access control | Sanctioned AI use depends on controlled access paths, not ad hoc personal accounts. | |
| Recommendation — Publish and enforce acceptable-use rules for generative AI tools. Allow work use only through access-controlled AI services. | ||
Practitioner Guidance
What to prioritise: Treat sanctioned access, not model output quality, as the first control question. If the organisation cannot prove which account path was used and what policy governed it, the interaction should be considered high risk until proven otherwise.
What to verify: Check whether employees can reach ChatGPT through an approved enterprise tenancy, whether logging is retained, and whether data-entry rules are enforced in practice. A policy without a managed entry point is usually aspirational, not operational.
Common mistake: Focusing only on “AI hallucinations” while ignoring personal-account use. In work settings, the larger failure is often ungoverned disclosure and lack of auditability, not the quality of the answer returned.
Practitioner takeaway: If you cannot observe, govern, and evidence the use path, you do not really control workplace ChatGPT use, you only hope employees will use it safely.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on October 8, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org