Topic starter
20/08/2026 5:23 pm
TL;DR: Generative AI safety fails when organisations rely on model behaviour instead of controlling access, grounding, and monitoring, according to Fiddler’s discussion with Peter Norvig. Systems need separate paths for creative output and factual output, with logging and testing on both.
Editorial analysis by NHI Mgmt Group, based on content published by Fiddler: “AI Safety in Generative AI”.
Key questions
Q: Why do open APIs change the AI safety problem for enterprises?
A: Open APIs turn AI usage into something the organisation can observe and control, which is better than uncontrolled model distribution.
Q: What are the signs that a generative AI programme is creating unmanaged risk?
A: Warning signs include heavy dependence on external partners, unclear ownership for AI use cases, weak controls over sensitive data, and rapid deployment without compliance review.
Q: How should organisations govern generative AI when output can be both creative and factual?
A: Separate creative generation from factual reporting and require grounding, provenance, or review wherever the output influences decisions, legal claims, or external communication.
Practitioner guidance
- Define separate access paths for factual and creative AI use Route factual outputs through grounded systems with approved knowledge sources, and keep creative generation in a distinct policy lane with tighter review where needed.
- Constrain API access and usage scope Limit who can invoke model endpoints, which prompts and tools they can reach, and what rate or volume thresholds trigger review or suspension.
- Instrument AI logs for misuse detection Capture prompt, tool, retrieval, and output metadata so abuse, unsafe synthesis, and unusual consumption patterns can be investigated quickly.
Bottom line: Generative AI safety fails when organisations rely on model behaviour instead of controlling access, grounding, and monitoring.
Glossary terms: Output Grounding, Generative AI Security Risk, Red Teaming, API-mediated Access
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 2 days ago by NHI Mgmt Group
20/08/2026 6:13 pm
Control, not trust, is the correct operating model for generative AI. Safety claims that rely on model intent or vendor assurances do not hold up once systems are exposed through APIs, embedded in workflows, or connected to internal knowledge sources. The control surface is the combination of access, logging, tooling, and output policy, and that surface has to be governed explicitly. For practitioners, AI safety should be treated as a control architecture problem, not a model confidence problem.
A few things that frame the scale:
- According to Gartner, worldwide spending on generative AI is set to reach $644 billion in 2025, a nearly 77% year-over-year increase.
A question worth separating out:
Q: What should security teams do when generative AI is being used to synthesise harmful information?
A: Tighten access, isolate high-risk use cases, and add monitoring that can identify misuse patterns before they spread. The response is not just blocking content after the fact. It is reducing who can reach the system, what the system can access, and how quickly suspicious behaviour is detected.
👉 Read our full editorial: Generative AI safety turns on access control and output grounding
This post was modified 2 days ago by NHI Mgmt Group