Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

LLM application security: are code and runtime controls catching up?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 13011
Topic starter  

TL;DR: LLM risk is increasingly concentrated in the application layer, where AI-generated code, hallucinated packages, malicious MCP servers, and shadow AI create exposure that model guardrails cannot stop, according to Aikido. Its cited figures show 24% of production code is AI-generated and 69% of organisations have already found vulnerabilities in AI-written code, making application security, not model-only protection, the decisive control plane for AI development.

NHIMG editorial — based on content published by Aikido: Top LLM security tools to protect AI applications

By the numbers:

Questions worth separating out

Q: How should security teams govern AI-generated code in production environments?

A: Security teams should treat AI-generated code as normal production code with extra provenance risk.

Q: Why do AI coding agents increase supply-chain risk?

A: AI coding agents can choose tooling, install packages, and edit lock files in ways that may bypass the controls humans expect in a managed environment.

Q: What do teams get wrong about LLM-as-judge guardrails?

A: They often assume the judgment layer is a replacement for other controls.

Practitioner guidance

  • Implement IDE-time scanning for AI-generated code Move SAST and secrets detection into the IDE so untrusted input, hardcoded credentials, and unsafe prompt patterns are caught when code is written, not after it enters a pull request.
  • Block hallucinated and malicious packages before install Use package-firewall controls and dependency allowlists to stop slopsquatted npm or Python packages at install time, and tie exceptions to explicit approval workflows.
  • Treat MCP servers as privileged integrations Inventory every MCP server, extension, and connected tool that can reach prompts, files, or secrets, then require approval and monitoring before those tools are trusted in developer environments.

What's in the full article

Aikido's full blog covers the operational detail this post intentionally leaves for the source:

  • Side-by-side feature comparison across Aikido, Snyk, Semgrep, Endor Labs, and Wiz for AI application security workflows.
  • Hands-on explanation of how Aikido's IDE scanning, package blocking, and runtime tracking are positioned in the development lifecycle.
  • Detailed treatment of PromptPwnd, slopsquatting, and malicious MCP server threats that underpin the product evaluation.
  • Tool-by-tool coverage notes for teams deciding whether they need code-side, supply-chain, or environment-level controls.

👉 Read Aikido's comparison of top LLM security tools for AI applications →

LLM application security: are code and runtime controls catching up?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 3 months ago
Posts: 12595
 

Application-layer LLM security is now a governance requirement, not a niche AppSec concern. The article correctly shifts the focus from model behaviour to the code, tools, and data paths that surround LLMs. That is where most real-world exposure appears, including secrets in workflows, unsafe prompt construction, and dependency risk. For identity teams, the lesson is direct: if an AI system can call tools or access credentials, it is operating inside an access-control boundary, not outside it.

A question worth separating out:

Q: How should organisations govern shadow AI without blocking legitimate use?

A: Start with approved-use policy, tool inventory, and data classification. Then require that any AI system handling internal information has named owners, logged access, and defined credential paths. The goal is not prohibition, but visibility and control. If a tool cannot be inventoried or monitored, it should not process sensitive data.

👉 Read our full editorial: LLM application security is shifting to code, supply chain, and runtime



   
ReplyQuote
Share: