Join our Newsletter — 33% off our NHI Course

AI agent skills and runtime scanning gaps: are your controls enough?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 18346
Topic starter  

TL;DR: AI agent skill marketplaces now carry the same supply chain risk pattern as software packages, but static code checks and LLM-based review both miss malicious behavior that only appears at runtime, according to Permiso Security. Runtime detonation is becoming the deciding control because executable trust cannot be inferred from code alone.

Editorial analysis by NHI Mgmt Group, based on content published by Permiso Security: “Introducing SandyClaw - The First Dynamic Sandbox for AI Agent Skills and Prompts”.

By the numbers:

  • One industry audit found over 1,100 malicious skills on a single marketplace.

Key questions

Q: What fails when AI agent skill scanners only inspect source code?

A: They miss behaviors that appear only during execution, such as unauthorized network calls, file writes, credential access, or data exfiltration.

Q: Why do AI agent skills create supply chain risk in enterprise environments?

A: AI agent skills create supply chain risk because they can look harmless in review while delivering malicious behavior only when executed.

Q: How should security teams validate AI agent skills before installation?

A: They should execute each skill in a controlled sandbox with real agent context and inspect actual behaviour, not just source code.

Practitioner guidance

  • Define runtime approval gates for agent skills Require every downloadable skill to pass a controlled execution check before it can be installed into an agent workflow or internal skill registry.
  • Instrument skill execution with behavioral evidence Capture LLM actions, network calls, DNS resolution, file access, and environment variable reads so security reviewers can verify what the skill did, not what it claimed to do.
  • Separate skill review from source review Do not treat static code analysis or LLM judgment as a complete trust decision when the skill can act on live credentials, APIs, or internal services.

Bottom line: AI agent skills inherit the access of the agent that installs them, so marketplace distribution becomes an identity-backed supply chain risk.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 17926
 

Runtime validation is now the governing control for AI agent skills: static review can reduce obvious risk, but it cannot prove what a skill will do once it inherits live permissions. The decisive question is no longer whether the code looks suspicious, but whether the skill produces unsafe behavior when executed against real tools and data. Practitioners should treat execution evidence as the primary trust signal.

A few things that frame the scale:

  • 92% of organisations expose NHIs to third parties, raising concerns about supply chain security, according to the Ultimate Guide to NHIs.

A question worth separating out:

Q: What breaks when organisations rely only on static analysis for AI agent skills?

A: Static analysis breaks down because it can only inspect what is written, not what happens at runtime. Hidden links, delayed instructions, self-reinstalling behavior, and remote payload retrieval can all remain invisible until the skill is executed. That creates false confidence, delayed detection, and missed opportunities to block persistence or exfiltration before deployment.

👉 Read our full editorial: Static skill scanners miss runtime AI agent risk



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.