How should security teams prepare for GenAI adoption without assuming model prompts are the only risk?

GenAI adoption risk starts with capability, not just prompting

Security teams often over-focus on prompt injection because it is visible and easy to demo, but that view misses the larger control problem. GenAI systems can expose data, trigger actions, and propagate bad outputs across workflows even when prompts are well governed. The real question is what the model is allowed to read, retain, recommend, and execute across the surrounding application stack. For a governance lens on that broader risk surface, NIST AI 600-1 GenAI Profile is the more relevant reference than a prompt-only view. In practice, many security teams discover the highest-risk GenAI exposure only after a pilot is already connected to sensitive systems and business users have started relying on its outputs.

How GenAI controls work once the model is embedded in a workflow

Preparation should begin by mapping the full operating context of each use case: who can use it, what data sources it can reach, what actions it can influence, and where its outputs are consumed. A chatbot used for policy search has a very different risk profile from an assistant that can draft tickets, query internal repositories, or trigger downstream automation. The security boundary therefore sits around the application, the integration layer, the data permissions, and the business workflow, not only around the text prompt.

Good practice is to define access in layers. First, decide which data classes the GenAI system may retrieve. Then decide whether the model may merely suggest actions or whether any action requires explicit human approval. Finally, test whether the system can be induced to reveal restricted context, overstate confidence, or route users into unsafe decisions. That testing should include indirect abuse, such as content that causes the model to surface sensitive information from a connected source or produce instructions that downstream users treat as authoritative.

• Inventory the use case, the model provider, the connected systems, and the business owner.
• Separate read access, write access, and execution authority so each can be reviewed independently.
• Classify the data the system can reach, not only the data users type into the prompt.
• Test outputs for harmful advice, leakage, policy bypass, and unsafe automation paths before release.

This approach aligns with broader cyber governance expectations because it treats GenAI as part of a controlled service chain. The general lesson is that prompt controls are necessary but insufficient, especially when the model sits inside a process that can move data or initiate work. Where organisations cannot explain the model’s reach in plain terms, they do not yet understand the real control surface. For a general security-governance baseline, the NIST Cybersecurity Framework 2.0 is useful for structuring ownership, monitoring, and response around the service as a whole.

Where this guidance breaks down is when the GenAI system is effectively a prototype with no stable data boundaries, no named owner, or no reliable way to test downstream actions before deployment.

Where prompt-centric thinking breaks down in real deployments

Tighter prompt governance often increases operational overhead, requiring teams to balance user freedom against data leakage, workflow friction, and approval latency. That tradeoff becomes more visible when teams move from isolated testing to production integrations. In many deployments, the hard problems are not the words in the prompt but the permissions attached to the surrounding account, the connectors, and the output channel.

One common variation is a read-only assistant that still creates risk because it can aggregate privileged context from multiple sources and present it to users who would not normally have that picture. Another is an agentic workflow where the model does not directly hold credentials but can influence a tool that does. In those cases, the useful control question is not whether the prompt is safe, but whether the end-to-end decision chain can be trusted. There is not full industry consensus yet on how much autonomy is acceptable for GenAI systems that touch production data, so organisations should document their tolerance rather than assume a universal pattern exists.

Edge cases also include vendor-hosted features that change over time, where a previously harmless capability becomes risky once a connector, memory function, or action plugin is enabled. Those changes should be treated as control boundary changes, not as minor product updates. If the team cannot tell when the system’s effective privileges changed, the prompt review process is no longer the main issue.

Practitioner Guidance

What to prioritise: Establish the model’s reach before scaling adoption. Security teams should focus first on data access, actionability, and ownership because those are the controls most likely to determine whether the system becomes a controlled assistant or an uncontrolled decision layer.

What to verify: Confirm that each use case has a named business owner, a defined data scope, and a clear rule for when human approval is required. If any of those three are missing, the deployment should be treated as a higher-risk exception rather than a normal release.

What to measure: Track how often the system is connected to new sources, given new permissions, or allowed to influence downstream actions. Those changes are often a better signal of rising risk than prompt-volume metrics or user satisfaction scores.

Practitioner takeaway: The safest GenAI programmes are the ones that can explain and constrain the system’s real authority, not just its wording.

Related resources from NHI Mgmt Group

• How should security teams register identity risk assessments in a community model without creating access friction?
• How should security leaders prepare for agentic AI adoption without creating new identity and data access risk?
• How should security teams implement zero-trust controls for enterprise AI systems without assuming the model itself is trustworthy?
• How should security teams reduce the risk of phishing-driven data exposure without assuming credential compromise has occurred?