Topic starter
02/08/2026 12:31 pm
TL;DR: Teams that ship LLM features often blur AI safety and AI security, but the article shows why that creates exploitable gaps, from autonomous agents deleting production data to prompt injection and hidden instructions, according to Promptfoo and cited incident reports. The practical lesson is that helpful outputs and secure runtime behaviour need separate testing, controls, and accountability.
Editorial analysis by NHI Mgmt Group, based on content published by Promptfoo: “AI Safety vs AI Security in LLM Applications: What Teams Must Know”.
Key questions
Q: How should security teams test LLM applications for runtime security failures?
A: Test the full runtime path, not just the model in isolation.
Q: Why do APIs become a bigger security issue when AI agents consume them?
A: APIs become a bigger security issue because agents do not compensate for ambiguity, broken contracts, or overbroad scopes the way humans often do.
Q: What are the signs that prompt injection is bypassing AI governance?
A: Warning signs include unexpected tool calls, answers that ignore prior policy, sudden disclosure of sensitive fields, and citations that look valid but do not support the output.
Practitioner guidance
- Separate safety and security testing programs Run model-behavior tests for harmful output and security tests for prompt injection, data exfiltration, and unauthorized tool use as distinct control families.
- Constrain agent tool privileges Scope each agent to the minimum database, API, and file-system permissions needed for its task, and treat those permissions as revocable credentials.
- Sanitize and label external content Classify incoming emails, files, calendar events, and web content as untrusted inputs before they enter model context or tool execution paths.
Bottom line: AI safety and AI security solve different problems, so one control stack cannot cover both failure modes.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
02/08/2026 1:28 pm
AI safety and AI security are governance problems with different control objectives. Safety is about constraining what a model outputs; security is about constraining what an adversary can make the system do. The article is right to separate them because one set of controls cannot compensate for the other when the model has tools, memory, and external data access. The practitioner conclusion is to design two test plans, two risk registers, and two ownership models.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: Should organisations treat AI as an application or as an identity?
A: Treat it as an identity when the AI can access data, invoke tools, or participate in workflows that affect business systems. That framing makes least privilege, just-in-time access, and lifecycle governance relevant. If you keep treating it only as an application, you will miss the access and delegation behaviours that actually create risk.
👉 Read our full editorial: AI safety and AI security are different problems with different controls