Topic starter
07/06/2026 8:23 pm
TL;DR: Open source models are closing the capability gap with closed systems and pushing more organisations toward in-house inference as monthly spend rises into five figures, according to WorkOS's interview with Baseten. The governance question is no longer whether AI workloads will scale, but which identity, access, and infrastructure controls will govern them when they do.
Editorial analysis by NHI Mgmt Group, based on content published by WorkOS: “Baseten is betting big on open source models”.
Key questions
Q: How should security teams govern in-house AI inference workloads?
A: Security teams should govern in-house AI inference workloads as non-human identities with scoped permissions, named ownership, and lifecycle controls.
Q: Why does open source model adoption change identity governance for AI platforms?
A: Because the control boundary moves from an external API provider to the enterprise's own runtime.
Q: What breaks when AI workloads scale without lifecycle controls?
A: When AI workloads scale without lifecycle controls, old credentials and broad privileges tend to remain in place after the system changes.
Practitioner guidance
- Define ownership for inference runtime access Assign a named owner for model deployment, GPU access, and orchestration permissions so AI runtime changes are governed like any other privileged production path.
- Inventory the non-human identities behind AI workloads List the service accounts, tokens, and deployment credentials that can start, stop, or modify inference workloads, then remove any unnecessary standing access.
- Separate development and production model access Keep experimentation, tuning, and production inference on distinct access paths so a lower-trust testing identity cannot modify live workloads.
Bottom line: Open source model quality is now close enough to closed systems that infrastructure and governance decisions are driving more AI workloads back in-house.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 3 days ago by NHI Mgmt Group
07/06/2026 10:15 pm
Inference governance is becoming a workload identity problem, not just a model-selection problem. Once organisations bring AI workloads in-house, the security question shifts to which identities can deploy, tune, and execute those models. The governance boundary moves from the SaaS provider to internal GPU, orchestration, and deployment layers. Practitioners should treat inference platforms as privileged production systems with explicit ownership and control.
A few things that frame the scale:
- Software supply chain attacks were projected to cost organisations $60 billion in 2025.
A question worth separating out:
Q: What is the difference between external AI APIs and in-house inference governance?
A: External APIs shift much of the runtime control to the provider, while in-house inference brings deployment, execution, and access decisions into the enterprise. That means the organisation must manage who can operate the model, who can change it, and which non-human identities have privileged access. The governance burden moves inward with the workload.
👉 Read our full editorial: Open source model adoption is shifting AI workloads in-house
This post was modified 3 days ago by NHI Mgmt Group