Notifications
Clear all
Topic starter
20/08/2026 5:23 pm
TL;DR: AI safety in generative AI depends less on chatbot guardrails than on whether organisations can bound misuse, separate creativity from factual output, and keep human control over systems that can synthesize harmful or misleading information, according to Fiddler’s discussion with Peter Norvig. The practical challenge is governance, testing, and monitoring, not just model capability.
NHIMG editorial — based on content published by Fiddler: AI Safety in Generative AI
Questions worth separating out
Q: How should security teams govern generative AI tools that connect to core systems?
A: Treat them as non-human identities with lifecycle, access, and telemetry requirements.
Q: Why do agentic AI systems require continuous red teaming?
A: Agentic systems change when models are retrained, prompts are updated, retrieval sources shift, or new integrations are added.
Q: What do organisations get wrong about AI safety and hallucinations?
A: They often treat hallucination as a purely content-quality issue when it is also a control problem.
Practitioner guidance
- Define AI system access boundaries Map every model, prompt pipeline, retrieval layer, and tool connector to the data and actions it is allowed to use.
- Run continuous adversarial testing Schedule recurring red-team exercises that probe unsafe outputs, data leakage, instruction bypass, and tool misuse across real production flows.
- Separate factual and creative modes Use governed retrieval, source citation, and policy routing so high-stakes answers cannot rely on unconstrained generation.
What's in the full article
Fiddler's full blog covers the operational detail this post intentionally leaves for the source:
- The on-demand fireside chat with Peter Norvig on preserving human control in generative AI systems
- The article's discussion of AI fairness, surveillance concerns, and the need for broader stakeholder impact analysis
- The section on separating creativity from factual reporting in LLM outputs, including source attribution and knowledge-base use
- The broader responsible-AI argument about internal self-regulation, third-party certification, and API restrictions
👉 Read Fiddler's analysis of AI safety in generative AI →
AI safety in generative AI: what controls are actually missing?
Explore further
20/08/2026 6:13 pm
AI safety becomes an identity governance issue the moment a model can act beyond a single prompt. Once a generative system can retrieve data, invoke tools, or shape downstream decisions, it starts behaving like a governed workload rather than a standalone application. That means access scope, accountability, and runtime boundaries matter more than model size alone. Practitioners should treat AI safety as part of IAM and operational governance, not as a separate ethics exercise.
A question worth separating out:
Q: How do human review controls reduce AI safety risk?
A: They work only when the review point is embedded where the risk occurs, not after the fact. Human approval should sit before sensitive data release, privileged action, or external submission. If the system can complete the task before review happens, the control is advisory rather than preventive.
👉 Read our full editorial: AI safety in generative AI depends on control, not trust
