In-context learning shows why prompt control is not enough

At a glance

What this is: This is a beginner’s guide to in-context learning in large language models, with the key finding that prompt examples can steer model behaviour at inference time without retraining.

Why it matters: It matters because IAM, NHI, and AI governance teams cannot rely on static access decisions alone when model behaviour changes inside a session based on the context it receives.

👉 Read Lakera's guide to in-context learning in large language models

Context

In-context learning is the ability of a pretrained language model to adapt its output from examples placed in the prompt, without changing model weights. For AI security and identity governance teams, the important point is that the system can shift behaviour at runtime even when the underlying model and its nominal permissions stay the same.

That creates a control problem for AI operations. If a model can infer patterns, reuse instructions, and respond differently based on in-session context, then policy based only on pre-approved access is incomplete. Governance has to account for what the model is exposed to, what it is induced to do, and what downstream systems it can influence during execution.

Key questions

Q: How should security teams govern prompts in production AI systems?

A: Treat prompts, examples, and retrieval context as governed production inputs. Review them for sensitive data, version them like application changes, and test how they affect model behaviour before deployment. The goal is to reduce the chance that transient context will steer outputs, leak information, or influence downstream actions in ways the security team did not intend.

Q: Why do AI models create governance risk even without retraining?

A: Because behaviour can change at inference time when the model sees new context, examples, or instructions. That means access decisions made before a session starts are not enough on their own. Practitioners need controls that address what the model can consume and do during execution, not only what it was permitted to access originally.

Q: What do security teams get wrong about prompt engineering?

A: They often treat prompt engineering as a purely quality or UX activity. In production, it also shapes security outcomes because it determines what context the model sees and how consistently it responds. If prompts are not reviewed, tested, and governed, they can become a hidden source of policy drift and data exposure.

Q: How do you know if an AI workflow is too dependent on context?

A: Look for workflows where small prompt changes create large output shifts, especially when sensitive data, code, or operational instructions are included. If the model’s behaviour is fragile, the workflow is overexposed to context risk. That is a sign the system needs tighter data minimisation, prompt testing, and stronger execution boundaries.

Technical breakdown

How in-context learning works without retraining

In-context learning, often called few-shot prompting, lets a model infer a task from examples embedded in the prompt. The model does not update weights, but it uses pretraining-derived latent patterns to predict the next token in a way that matches the demonstrated task structure. That is why a carefully written prompt can steer translation, classification, summarisation, or code generation without any fine-tuning. The mechanism is transient, but the effect is real during the active session, which makes it operationally useful and security relevant.

Practical implication: treat prompts, examples, and retrieved context as operational inputs that need governance, not as harmless text.

Why prompt engineering becomes a security control surface

Prompt engineering is the deliberate construction of instructions and examples to guide model behaviour. In security terms, that turns the prompt into a control surface, because a model can be pushed toward different outputs by changing context rather than permissions. The article also points to prompt brittleness, where small changes can alter results materially. That instability matters because it means a model’s behaviour is partly shaped by the quality, ordering, and wording of input, not just by its base configuration.

Practical implication: validate prompts as you would other production controls, because prompt drift can create behaviour drift.

Why in-context learning creates exposure for enterprise AI workflows

The article ties in-context learning to real-world use cases such as content generation, translation, sentiment analysis, and code-related tasks. Those workflows often sit close to sensitive data, so the model’s ability to infer patterns from context can become a privacy and security issue if prompts contain secrets, credentials, or policy-sensitive material. Once the model has seen enough examples, it may reproduce or generalise patterns that were never meant to leave the workflow boundary. That is a runtime data handling problem, not just a model quality problem.

Practical implication: minimise sensitive context in prompts and retrieval layers before those inputs reach production AI systems.

NHI Mgmt Group analysis

In-context learning creates a runtime policy problem, not just a model behaviour problem. The article shows that prompts can alter outputs without retraining, which means the security boundary moves from model weights to session context. That is especially relevant for AI systems that touch identity, data, or enterprise workflows, because what they are shown can materially change what they do. Practitioners should treat prompt context as governed input, not incidental text.

Prompt engineering is now part of the control plane for AI systems. When example selection, ordering, and wording shape output quality, the prompt becomes an operational dependency rather than a documentation artifact. This is where AI governance starts to overlap with NHI-style control thinking, because the system is acting on transient, runtime instructions that can steer downstream actions. The implication is that prompt design belongs in security review, change management, and testing.

Transient context: in-context learning works because the model can use temporary examples to change behaviour during a single session. That assumption was designed for static prompts and bounded tasks. It fails when the actor is a production AI workflow that can consume sensitive context, influence tools, and affect downstream systems before any human review occurs. The implication is that identity governance must account for session-scoped behaviour, not only provisioning-time access.

LLM security and NHI governance are converging around the same runtime question. If a model can be guided by context to produce different actions, then least privilege alone does not describe the real exposure. The decisive issue becomes how much sensitive information, tool access, and execution authority the system can combine during inference. Practitioners should align AI controls with identity lifecycle, secrets handling, and execution boundaries rather than treating the model as a passive text engine.

From our research:

• The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to The State of Secrets in AppSec.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap, according to The State of Secrets in AppSec.
• For broader NHI governance context, see the NHI Lifecycle Management Guide for provisioning, rotation, and offboarding discipline.

What this signals

Transient context is the new exposure window. In-context learning makes it harder to think about AI security as a static allow or deny problem, because behaviour can shift within a single session. For practitioners building governance around model use, the operational question is whether the prompt path is treated with the same control discipline as other runtime inputs, especially when identity, secrets, or workflow decisions are nearby.

The broader signal is that AI programmes are moving closer to identity and access governance whether teams planned for it or not. Once prompts can shape actions, the security model must cover data minimisation, execution boundaries, and reviewable change control. That is why NHI-style lifecycle thinking and AI workflow governance increasingly overlap, even when the model itself remains non-autonomous.

With 27 days to remediate a leaked secret in our research, the mismatch between exposure and response is already well established in machine identity programmes. In-context learning extends that lesson into AI workflows: if sensitive context enters the session, the damage can happen long before the control team finishes reviewing it.

For practitioners

• Classify prompts as governed inputs Review prompt templates, examples, and retrieved context the way you review other production control inputs. Remove secrets, credentials, and policy-sensitive material before those values enter the inference path.
• Test for behaviour drift under prompt variation Run structured evaluations that change example order, wording, and context length to see where the model’s output changes materially. Use the results to identify brittle workflows before they reach users or downstream systems.
• Separate sensitive data from prompt context Keep confidential data out of prompt text wherever possible and rely on controlled retrieval, redaction, and data minimisation instead. This reduces the chance that the model will generalise from material it should never have seen.
• Put AI prompt changes under change control Treat prompt edits like production changes, with review, versioning, and rollback. If a prompt can change model behaviour, it deserves the same discipline as other runtime policy inputs.

Key takeaways

• In-context learning lets pretrained models change behaviour from prompt examples alone, which moves security attention from training pipelines to runtime context.
• The practical risk is not retraining, but the way prompts, retrieved data, and instructions can steer outputs and expose sensitive material during a live session.
• Identity and AI governance teams should treat prompt design, context minimisation, and change control as production controls, not optional engineering hygiene.

Key terms

• In-context learning: A model’s ability to adapt its output from examples placed in the prompt without changing its underlying weights. The behaviour is temporary but operationally meaningful, because the session context can steer what the model produces for the duration of inference.
• Prompt engineering: The practice of writing and structuring prompts so a language model produces more reliable outputs. In security terms, it also becomes a control activity because the prompt determines what context the model sees, how it interprets tasks, and how much risk is carried into the session.
• Runtime context: The information a model receives while it is actively generating output, including instructions, examples, retrieved data, and nearby text. Runtime context matters because it can alter behaviour without retraining, making it a live governance surface rather than a passive input field.

What's in the full article

Lakera's full article covers the conceptual mechanics this post intentionally leaves at a higher level:

• A step-by-step explanation of Bayesian inference and how latent concepts are inferred from prompt examples.
• A deeper walkthrough of few-shot, one-shot, and zero-shot in-context learning patterns.
• Discussion of prompt engineering as a practical technique for shaping model output quality.
• Additional examples of real-world LLM applications and research directions that influence implementation choices.

👉 The full Lakera article covers the mechanisms, variants, and research references behind in-context learning.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or operational governance in your organisation, it is worth exploring.