GenAI and LLM Application Security: Executive Briefing

Organisations are putting large language model (LLM) applications in front of staff and customers faster than they are deciding who owns the risk when one leaks data, follows a planted instruction or says something the business cannot stand behind. This briefing is for executives, product owners and CISOs approving a specific GenAI application: a customer chatbot, an internal assistant over company documents, or an LLM feature inside an existing product. It covers four deployment decisions that sit with leadership: what data the application can see, how it handles untrusted content, what the model supplier's terms allow, and who is accountable for its output. Board-level oversight of AI security as a whole is covered in the Board Briefing: AI Security, data oversharing in the Data Security and AI Executive Briefing, and agent identity in the Board and CISO Briefing.

The risk in one paragraph

An LLM application combines a model you probably do not control, instructions you wrote, data you connected and text supplied by users or the outside world, and the model cannot reliably tell these apart. The OWASP Top 10 for LLM Applications 2025 puts prompt injection first (LLM01:2025) and sensitive information disclosure second (LLM02:2025): content the application reads can change what it does, and anything it can reach can end up in an answer, a log or a supplier's systems. Add the supplier's own terms on retention and training, the cost of an unprotected API key, and the reputational effect of a confident wrong answer to a customer, and an LLM launch becomes a set of business decisions, not only a technical one. None of these risks is fixed by telling the model to behave. They are reduced by limiting what the application can see and do, contracting carefully, testing before release and watching it in production.

Why this is a board-level issue now

Accountability does not transfer to the model supplier: the UK Code of Practice for the Cyber Security of AI (January 2025) assigns duties to the organisation operating the system as well as to its developer, and its Principle 4 is "Enable human responsibility for AI systems".
Data protection still applies: the ICO's guidance on AI and data protection expects accountability and a data protection impact assessment (DPIA) where AI processes personal data; the guidance is under review following the Data (Use and Access) Act, so check its current status before relying on detail.
Allied agencies set expectations: the joint NSA, CISA, FBI, NCSC-UK and partner guidance Deploying AI Systems Securely (April 2024) asks deployers to name who is accountable for AI cyber security, document risk acceptance and require a threat model from the developer.
Incidents are already public: a hidden instruction in a calendar invite led an AI assistant to summarise private meetings into an event an outsider could read (see Gemini AI Calendar Prompt Injection 2026), and an exposed AI provider database leaked chat histories and API keys (see DeepSeek Database Exposure 2025).

Four deployment decisions

Decision	What can go wrong	OWASP LLM 2025	What leadership should require
What data the application can reach	Answers expose personal, confidential or other users' data; system prompts reveal internal logic or credentials	LLM02 Sensitive Information Disclosure; LLM07 System Prompt Leakage	Retrieval that respects each user's existing permissions; no secrets in prompts; data classification before connection
How untrusted content is handled	Text in an email, web page, document or user message redirects the application	LLM01 Prompt Injection	Treat all external content as untrusted; no consequential action without approval; output checked before it reaches other systems (LLM05)
What the supplier's terms allow	Prompts retained, reviewed or used for training; data stored outside approved regions; model changed without notice	LLM03 Supply Chain	Contracted positions on retention, training use, location, sub-processors, incident notice and model change
Who owns the output	Confident wrong answers given to customers or used in decisions; runaway cost from abuse	LLM09 Misinformation; LLM10 Unbounded Consumption	A named business owner; human review where answers carry legal or financial weight; rate and spend limits

For the full list and technical detail, see the OWASP Top 10 for LLM Applications Guide and the LLM Security Guide.

Supplier terms to settle before launch

Training use: whether prompts, files and outputs can train or improve the supplier's models, and whether that differs between consumer, business and API offerings.
Retention and review: how long prompts and outputs are kept, including for abuse monitoring, and whether supplier staff can read them.
Location and sub-processors: where data is processed and stored, and which third parties host or serve the model.
Change notice: how much notice you get before a model version is changed or retired, since behaviour and safety can shift between versions.
Incident terms: how quickly the supplier will tell you about a security incident affecting your data, and what logs it will share.

The AI Vendor Security Questionnaire Template turns these into supplier questions.

Five questions for the board to ask

Which LLM applications are live or in build, and who owns each one? Expect a register naming a business owner, the model and supplier, the data connected and the users served.
Can any application show a user data they could not otherwise see? Expect evidence that retrieval honours existing permissions and that this was tested.
What happens if the application reads a malicious instruction? Expect an answer about what the application is allowed to do, not about filters; actions that send, change or delete data should need a person to approve them.
What do our supplier contracts say about retention, training and model change? Expect the actual terms for each production application, not a general statement.
How would we know it was misbehaving, and who would switch it off? Expect logging of prompts and outputs within data protection limits, monitoring for abuse and cost spikes, and a named person with authority to disable it.

Red flags in management answers

"The system prompt tells it not to reveal that." Instructions to the model are not a security control.
"The supplier handles security." The supplier secures its platform; your data, connections, keys and users remain your responsibility.
"We are on the enterprise plan, so our data is safe." Ask which terms that plan actually contains.
No answer on who approved go-live or who can turn the application off.

What good looks like

Every LLM application has a named owner, a threat model and a recorded go-live decision, renewed when the model, data or capabilities change.
Applications see only what the requesting user may see, and secrets never sit in prompts.
Consequential actions require human approval; outputs are validated before reaching other systems.
Supplier terms on training, retention, location and change notice are known and acceptable for each data type.
Applications are tested before launch, including adversarial testing, and monitored in production for abuse, leakage and cost.
API keys for model services are treated as privileged non-human identities: stored in a secrets manager, scoped and rotated.

Where to invest first

An application register and owner model: cheap, and the base for every other control.
Permission-aware retrieval and data clean-up: the most common route to leakage is an application connected to content that was already overshared.
Pre-launch testing: adversarial tests for prompt injection and data disclosure on each customer-facing release. See the AI Red Teaming Guide.
Model service key protection: stolen LLM API keys have been widely abused to run models at victims' expense (see LLMjacking 2024 to 2026), so move keys into a secrets manager and set spend alerts.
Logging and a kill switch: prompt and output logs with appropriate retention, and a tested way to disable each application quickly.

Metrics to track

Metric	What it shows	Direction
Share of LLM applications with a named owner, threat model and go-live approval	Governance coverage	Towards 100%
Share of production applications with reviewed supplier terms on training and retention	Contractual control	Towards 100%
Open high-severity findings from pre-launch adversarial testing	Release quality	Down, none at launch
Data disclosure or prompt injection incidents, by application	Control effectiveness	Down
Model service API keys outside a secrets manager or older than the rotation period	Credential hygiene	Towards zero
Time to disable an application in a test	Readiness to contain	Down

A 90-day plan

Days 1 to 30: build the register of LLM applications and owners; collect supplier terms for each; move model API keys into a secrets manager; confirm who can disable each application.
Days 31 to 60: threat model the highest-risk applications using the AI Model Threat Model Template where you host or tune models; test retrieval permissions; require approval for consequential actions; run adversarial tests on customer-facing applications.
Days 61 to 90: turn on prompt and output logging with agreed retention; add cost and abuse alerts; write the go-live standard into policy; report the metrics above to the executive committee.

Where it sits in the frameworks

Framework	Where LLM application security fits
OWASP Top 10 for LLM Applications 2025	LLM01 to LLM10, the shared vocabulary for application risks
NIST AI 600-1 (Generative AI Profile)	Companion to the AI RMF 1.0 for generative AI risk management
UK Code of Practice for the Cyber Security of AI	13 principles, including threat evaluation (3), human responsibility (4), supply chain (7) and monitoring (12)
NCSC Guidelines for Secure AI System Development	Secure design, development, deployment, and operation and maintenance
NIST CSF 2.0	GV.OC and GV.RR for context and accountability; GV.SC for supplier risk

Standards and references

How NHI Mgmt Group can help

We provide independent identity and non-human identity assessments and executive briefings, including reviews of the API keys, service accounts and OAuth grants that LLM applications use to reach models and company data. Our NHI Foundation Level Training Course includes a section on agentic AI. Contact us to arrange a briefing.

GenAI and LLM Application Security: An Executive Briefing on Deployment Risk