Experiment

An experiment is a controlled run of a prompt against a dataset so teams can measure performance before release. It captures outputs, applies the scorer, and creates a versioned record of results. This lets practitioners compare prompt revisions side by side and see where accuracy improved or regressed.

Expanded Definition

In AI operations, an experiment is more than a one-off test. It is a repeatable evaluation event that ties a specific prompt, dataset, scorer, and configuration to a versioned result set so teams can compare changes with evidence rather than intuition. For NHIMG, the key distinction is between informal prompting and a governed experiment: the latter preserves traceability, supports reproducibility, and creates a defensible audit trail for prompt engineering decisions.

Usage in the industry is still evolving, and definitions vary across vendors. Some platforms use experiment to mean prompt testing, while others include model routing, retrieval settings, safety filters, or downstream task variants. A useful reference point is the governance orientation of the NIST Cybersecurity Framework 2.0, which emphasises repeatable risk management and accountability even when applied outside traditional IT assets. The strongest definition of experiment is one that can be rerun, compared, and explained. The most common misapplication is treating ad hoc prompt trials as experiments, which occurs when teams do not freeze the dataset, scorer, or model version.

Examples and Use Cases

Implementing experiments rigorously often introduces dataset governance overhead, requiring organisations to weigh faster iteration against the cost of controlling inputs, outputs, and evaluation criteria.

• A product team runs two prompt revisions against the same labeled dataset to determine whether the new wording improves answer relevance without increasing refusal rates.
• An AI security team uses experiments to compare a base prompt with and without retrieval augmentation, then reviews whether grounded responses reduce hallucinations under the same scorer.
• A governance team records each experiment in a change log so later reviewers can see which prompt, model, and safety settings produced a specific output set.
• A red-team workflow runs repeated experiments against adversarial inputs to see whether a prompt change weakens guardrails or expands unsafe completion patterns.
• A compliance or quality team uses experiment results to justify promotion, rollback, or further tuning before a prompt is released to production.

For teams building evaluation discipline, the experiment concept aligns with the broader control mindset reflected in the NIST Cybersecurity Framework 2.0: define the process, measure the outcome, and retain evidence that supports decision-making.

Why It Matters for Security Teams

Experiments matter because they turn prompt and model changes into governed evidence. Without them, teams often confuse a perceived improvement with a measurable one, then ship changes that degrade answer quality, widen exposure to prompt injection, or weaken safety behavior under real user traffic. For identity and agentic AI use cases, that matters even more: a poorly controlled experiment can mask changes in tool use, access paths, or retrieval behavior that alter what an agent is allowed to see or do.

Security teams should care about whether experiments are versioned, whether scorers are consistent, and whether the same test conditions are reused across releases. Those details determine whether a result is meaningful or just noise. When experiments are tied to security-sensitive workflows, they also support change management, incident review, and post-deployment validation. Organisational blind spots often surface only after a bad release, at which point experiment records become essential for explaining what changed and why it failed.

Related resources from NHI Mgmt Group

• Experiment Tracking