Topic starter
01/08/2026 8:58 pm
TL;DR: Hallucinated packages, malicious MCP servers and shadow AI push risk into LLM application security, with Aikido citing 24% AI-generated production code and 69% of organisations finding vulnerabilities in AI-written code. Creation-time, install-time and usage-time controls matter most when model guardrails miss the surrounding workflow.
Editorial analysis by NHI Mgmt Group, based on content published by Aikido: “Top LLM security tools to protect AI applications”.
By the numbers:
- 24% of production code is now AI-generated, according to Aikido.
- 69% of organizations have discovered vulnerabilities in AI-generated code, according to Aikido.
- 20% reported a serious incident as a result, according to Aikido.
Key questions
Q: Why do hallucinated package references create real supply chain risk in AI agent workflows?
A: Hallucinated package names become dangerous when agents and humans trust them without verification.
Q: What are the signs that shadow AI is becoming a governance problem rather than a productivity aid?
A: The clearest signs are widespread use outside IT visibility, repeated sharing of corporate data with GenAI tools, and no consistent approval path for new applications.
Q: What fails when LLM security is limited to model guardrails?
A: Model guardrails can shape output, but they do not inspect the workflow files, package installs, extensions or secrets that surround the model.
Practitioner guidance
- Scan AI-generated code at the point of creation Run SAST and secret detection inside the IDE so untrusted prompt output is checked before it reaches a pull request or build pipeline.
- Block hallucinated and malicious packages at install time Enforce dependency controls that stop slopsquatted packages, suspicious registries and known-malicious artifacts before they enter the dependency tree.
- Inventory MCP servers and AI extensions on developer machines Treat AI tooling as part of endpoint governance and monitor which servers, plugins and extensions can influence coding assistants or access project context.
Bottom line: LLM application security is the control layer that governs code, dependencies, developer tools and runtime integrations around the model.
Glossary terms: LLM Application Security, Slopsquatting, Shadow AI, MCP Server
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 5 hours ago by NHI Mgmt Group
01/08/2026 9:05 pm
Application-layer AI risk is now the decisive control problem. LLM guardrails can constrain model behaviour, but they do not govern the code, dependencies, developer tools, or runtime integrations that actually move data and privileges. That shifts the practical centre of gravity to application security, where SAST, SCA, secrets detection and runtime inventory provide the real enforcement points. For practitioners, the lesson is that model safety without application governance is incomplete.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: How should teams govern privileged tokens in agentic AI workflows?
A: Treat any workflow that lets an agent act on untrusted input as a high-risk execution path. Privileged tokens should not be available by default, and the agent’s authority should be narrowed so a prompt injection or malicious document cannot turn ordinary context into secret access.
👉 Read our full editorial: LLM application security is shifting to code, supply chain, and runtime