Join our Newsletter — 33% off our NHI Course

What would it take for Merlin to catch unsafe tool calls?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 2 years ago
Posts: 20217
Topic starter  

TL;DR: Valid-looking agent tool calls can still be unsafe when their actions do not follow from user intent or execution context, according to Kontext. Semantic review shifts tool governance from syntax to whether the call is justified by the interaction.

Editorial analysis by NHI Mgmt Group, based on content published by Kontext: “Merlin: Context-aware Local Detection of Unsafe Agent Tool Calls”.

By the numbers:

  • Merlin reached 91.19% accuracy on 7,182 held-out, step-labeled tool calls.

Key questions

Q: Why do prompt injections create more risk than ordinary malicious text?

A: Because the text is not the end state.

Q: How should teams decide which agent actions need stronger review?

A: Prioritise actions that change credentials, move money, send messages, delete content, or expose data outside the organisation.

Q: What breaks when untrusted content can influence privileged tool use?

A: The control boundary breaks between interpretation and execution.

Practitioner guidance

  • Add semantic review before execution Score the proposed tool call against the user request, prior steps, and tool descriptions before the action is allowed to run.
  • Separate read access from act authority Allow agents to inspect low-trust content without automatically inheriting permission to act on instructions embedded in that content.
  • Classify high-impact actions differently Use stricter thresholds for credential changes, payments, deletions, and external message sends than for low-risk read operations.

Bottom line: Prompt injection is dangerous because it can transform ordinary content into an unauthorized agent action without breaking schemas or permissions.

Glossary terms: Prompt Injection (Agentic), Semantic Tool-call Safety, Execution-visible Context, Intent Drift

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →


This topic was modified 7 days ago by NHI Mgmt Group

   
Quote
(@mr-nhi)
Member Moderator
Joined: 6 months ago
Posts: 20707
 

Semantic tool safety is now an identity governance problem, not just an LLM safety problem. When a model can read untrusted content and then choose a privileged action, the governance question shifts from output correctness to delegated authority. That means the control plane has to judge whether the action is warranted in context, not merely whether the API request is valid. Practitioners should treat tool-call evaluation as part of access governance, not as a post-hoc content filter.

A few things that frame the scale:

  • 28% of secrets incidents now originate outside code repositories, in Slack, Jira, and Confluence, and are 13% more likely to be categorised as critical than code-based leaks, according to the State of Secrets Sprawl 2026.

A question worth separating out:

Q: What is the difference between schema validation and semantic validation for agent tool calls?

A: Schema validation checks whether the request is syntactically acceptable and matches the function contract. Semantic validation checks whether the proposed action actually follows from the user’s intent and the prior interaction history. For agent security, both are needed, but only semantic validation catches many prompt-injection cases.

👉 Read our full editorial: Prompt injection makes agent tool safety a semantic governance problem


This post was modified 7 days ago by NHI Mgmt Group

   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.