How should security teams use system cards when evaluating LLM deployments?

Using a system card as an evaluation checkpoint, not a brochure

System cards are most useful when security teams read them as deployment evidence: what the model was tested against, what failure modes were observed, what safeguards exist, and where the published assurances stop. That makes the card a starting point for control design rather than a substitute for independent review. For LLM deployments, that distinction matters because a model can be technically impressive and still be unsuitable for a given use case if the operating context introduces prompt injection, unsafe tool use, policy drift, or unacceptable data exposure. The NIST AI 600-1 Generative AI Profile is helpful here because it frames generative AI through risk management, not marketing claims.

Security teams usually get better outcomes when they ask what the card does not cover, rather than treating the published summary as a full assurance package. In practice, many security teams encounter gaps only after a model has already been connected to sensitive workflows, external tools, or high-trust user groups, rather than during the original model review.

What to look for in the card before you approve a deployment

A useful system card should tell you whether the deployment has been evaluated for the actual way you intend to use it. If your use case includes customer data, regulated content, code generation, autonomous actions, or tool access, the card should help you judge whether those conditions were in scope. If it does not, treat the card as partial evidence. Security teams should also look for the model’s stated limitations, known unsafe behaviors, refusal boundaries, and any guidance about prompt sensitivity, jailbreak resistance, or content filtering. Those details help determine whether compensating controls belong in the application layer, the workflow layer, or the user interface layer.

System cards are strongest when they support decisions about policy boundaries. A team may decide, for example, to block certain data classes, require human review for high-impact outputs, constrain the model to a narrow task, or isolate it from internal systems until additional testing is complete. Where the deployment involves tool use or action-taking, the card should be read alongside governance and abuse-prevention thinking, not in isolation. For agentic or tool-using systems, the OWASP Top 10 for Agentic Applications 2026 is useful because it highlights how model capability becomes risk when it is connected to execution authority.

• Check whether the evaluation matches your actual workload, data class, and user population.
• Confirm whether the card describes residual risk, not just successful benchmark results.
• Look for explicit statements about jailbreaks, hallucinations, unsafe tool use, and data leakage.
• Use the card to decide which safeguards must remain external to the model.

The guidance breaks down when the card is vague, outdated, or written for a different deployment pattern than the one you are assessing.

When system cards are helpful, and when they are not enough

Tighter scrutiny of system cards often increases review effort, but that overhead is justified when the model will touch sensitive information, make recommendations with real consequences, or trigger downstream automation. The tradeoff is simple: the more the model can affect business decisions or system state, the less acceptable it is to rely on a generic summary. Where the deployment is low-risk and heavily sandboxed, a concise card may be enough to document assumptions and accepted limits. Where the deployment is high-impact, the card should be treated as one input into a broader assurance process.

Industry consensus is still developing on how detailed these documents should be, so security teams should not assume that every vendor uses the same structure or level of candour. Some cards are strong on evaluation results but weak on operational constraints; others describe intended behaviour but omit failure analysis. The practical response is to compare the card against your own threat model, data handling rules, and rollout plan. If those three things do not line up, the card is informational, not decisive. The NIST AI Risk Management Framework remains relevant as a governance lens because it pushes teams to connect model claims to measurable risk treatment.

Security teams should also remember that a system card cannot validate runtime behaviour after integration. Post-deployment monitoring, abuse testing, access controls, and escalation paths still matter because model behavior can change once prompts, retrieval sources, users, and tools are added.

Practitioner Guidance

What to prioritise: Start by checking whether the system card covers your actual deployment pattern, not just the base model. If your use case adds retrieval, tools, sensitive data, or high-impact decisions, assume the card is incomplete until proven otherwise.

What to verify: Verify the card names the tested limitations, the residual risks, and the operating conditions under which the model is considered acceptable. If those details are absent, require compensating controls before approval.

Decision rule: Treat the card as sufficient for low-risk pilots with tight scope and no privileged actions, but not as standalone assurance for production use that can affect data, customers, or infrastructure.

Practitioner takeaway: The best use of a system card is to turn model claims into concrete deployment decisions; if it does not change scope, controls, or monitoring, it is not doing security work.

Related resources from NHI Mgmt Group

• How should security teams use LLM-based identity risk scoring in production?
• How should security teams govern LLM use when browser security is not enough?
• How should security teams secure LLM system prompts in production applications?
• How should security teams use ISO 27001 and SOC 2 when evaluating cloud identity providers?