LLM-assisted code generation still needs review and tests

At a glance

What this is: This is ngrok’s retrospective on building a browser-based Kubernetes port with LLMs, and its key finding is that manual review plus integration tests were essential to make the output trustworthy.

Why it matters: It matters to IAM and security practitioners because the same verification gap appears whenever AI is used to generate operational code, automation, or identity-adjacent infrastructure that must behave deterministically.

By the numbers:

• At time of writing, webernetes has 204 integration tests alongside 1,855 unit tests.
• The final week of the project consumed $1,811.64 in API-equivalent token cost.

👉 Read ngrok's retrospective on building webernetes with LLMs

Context

LLM-assisted development changes the economics of software creation, but it does not remove the need for correctness controls. When AI is used to port a complex system, the real risk is not only bad code generation, but also the false confidence that comes from code which looks similar while behaving differently.

That matters for identity and security programmes because the same pattern shows up in automation, policy engines, and infrastructure code. If teams use AI to accelerate privileged workflows, access logic, or orchestration components, they need independent verification rather than trust in surface-level similarity. The project is a useful example of a broader governance problem, not an unusual one.

For identity teams, the most relevant lesson is that human review and automated tests are complementary controls. Review catches semantic drift and missing logic, while tests prove that the generated output still behaves like the system it is meant to replace or emulate.

Key questions

Q: How should teams validate AI-generated mobile code before release?

A: Teams should validate AI-generated mobile code against mobile-specific security standards, then test authenticated flows that exercise token handling, local storage, and backend API access. A clean compile or successful QA pass is not enough. The release decision should depend on whether the generated code has been checked under realistic runtime and identity conditions.

Q: Why do AI-generated ports need behavioural testing, not just code review?

A: Because code can look correct while behaving differently at runtime. Behavioural tests expose missed edge cases, altered control flow, and runtime assumptions that static review often misses. This is especially important when the code manipulates state, lifecycle transitions, or privileges, where small defects can have outsized operational impact.

Q: What do teams get wrong about using LLMs for infrastructure or platform code?

A: They often assume the model’s speed reduces the need for assurance. In practice, faster generation increases verification burden because humans must inspect more output in less time. The mistake is treating plausible output as proof of correctness, when only tests and review can establish that confidence.

Q: Should organisations use AI-generated code in security-sensitive workflows?

A: Yes, but only with strict controls around review, testing, and ownership. For security-sensitive workflows, the key question is not whether AI can generate code, but whether the organisation can prove its behaviour, maintainability, and failure modes. Without that proof, the delivery gain is offset by hidden risk.

Technical breakdown

Why LLM code ports drift from the source implementation

When an LLM ports code from one language or runtime to another, it often preserves the shape of the original without preserving every behavioural edge case. Common failure modes include shortcuts that replace specialised data structures with simpler ones, invented helper functions that alter control flow, and omitted test cases that break coverage. In complex systems, those differences are easy to miss in a line-by-line review because the output may look plausible while still changing semantics. That is why similarity is not the same thing as equivalence.

Practical implication: treat generated code as untrusted until it passes behavioural checks against the source system.

How integration tests validate AI-generated system behaviour

Integration tests are the control that proves whether the ported code actually behaves like the real system in non-trivial scenarios. In this case, the same test code could run against both a real Kubernetes cluster and the browser-based implementation, which exposed mismatches in lifecycle handling, API behaviour, and runtime assumptions. This is more powerful than unit tests alone because it checks the interaction between components, not just isolated functions. For systems with orchestration, access control, or stateful workflows, that interaction layer is where subtle defects usually surface.

Practical implication: build a comparison harness against the source of truth before you trust AI-generated infrastructure or automation.

What browser-based Kubernetes simulation changes for developers

Webernetes is not a production cluster. It is a constrained simulation designed to reproduce enough Kubernetes behaviour for interactive teaching, demos, and experimentation. That constraint is the key architectural idea: fidelity where it matters, simplification where it does not. The browser runtime replaces system-level APIs with simulated equivalents for pods, networking, and controllers, which makes it useful as a teaching tool but not a deployment target. This distinction matters because AI-generated systems often fail when teams confuse a demonstrator with an operational environment.

Practical implication: define the intended operating boundary clearly so AI-generated simulations are never mistaken for production control planes.

NHI Mgmt Group analysis

AI-generated code increases verification debt, not just delivery speed. The article shows that LLMs can accelerate production of large code volumes, but the real burden shifts to review, testing, and behavioural comparison. That pattern is familiar in identity and security engineering, where fast automation can conceal subtle defects in policy logic, privilege handling, or workflow orchestration. The practitioner conclusion is simple: if the system matters, verification is part of the build, not an optional follow-up.

Behavioural equivalence is the control that matters when code is ported by AI. Surface similarity is not enough when the target system implements stateful orchestration, access decisions, or event-driven workflows. The article’s emphasis on side-by-side testing against Kubernetes is a useful model for any team using generated code in security-sensitive paths. The practitioner conclusion is to compare runtime outcomes, not just source text.

Human review remains necessary because LLMs optimize for plausible completion, not system fidelity. The article’s examples of shortcuts, invented helpers, and missing tests reflect a broader governance problem: generated code is shaped by pattern completion, not by accountability for correctness. That matters for IAM, PAM, and infrastructure automation, where small deviations can create persistent access or policy defects. The practitioner conclusion is to assign human owners to every AI-assisted change set.

Simulation tools like webernetes are useful, but only when their limits are explicit. A browser-based cluster can support education and experimentation, yet it should never be confused with production control. That distinction is important for identity-adjacent tooling too, because lab environments often mask lifecycle, latency, and integration issues that appear only in real deployments. The practitioner conclusion is to separate teaching value from operational assurance.

Verification debt: the growing gap between how quickly AI can generate systems and how slowly teams can prove those systems are correct. This is the named concept the article points to, even if it does not use the phrase explicitly. The faster the generation cycle, the more organisations must invest in tests, review gates, and reproducible harnesses. The practitioner conclusion is to budget for assurance at the same time you budget for generation.

What this signals

AI-assisted development is pushing verification earlier in the delivery cycle, which means security teams should expect more demand for testable controls, reproducible environments, and reviewable change history. The useful pattern here is not speed alone but assurance that scales with automation.

Verification debt: when teams use AI to accelerate code generation, the gap between output volume and confidence grows unless testing and review keep pace. That applies directly to identity-adjacent automation, where small logic errors can turn into privilege or workflow defects. Reference material such as the NHI Lifecycle Management Guide is useful when generated code touches provisioning, rotation, or offboarding logic.

Security leaders should treat AI-generated infrastructure the same way they treat any other high-change control plane: define the reference behaviour, prove equivalence, and bound the blast radius. The more the code affects trust decisions, the more the programme needs explicit assurance rather than informal confidence.

For practitioners

• Require dual verification for AI-generated code Make human code review and automated behavioural testing mandatory for every generated change set that touches orchestration, policy, or access logic. Review is the only reliable way to catch semantic drift, while tests prove the output still behaves as intended.
• Create source-of-truth comparison harnesses Run the same test cases against the generated system and the reference implementation so you can detect mismatches in state handling, lifecycle transitions, and API responses. Use a real runtime when the code is intended to emulate production behaviour.
• Separate simulation from production controls Document which parts of an AI-built environment are educational or illustrative and which parts are operationally trusted. Browser-based demos, teaching clusters, and prototypes should never inherit production assumptions about security, durability, or identity governance.
• Track verification cost as part of delivery cost Measure review effort, test creation effort, and compute spend alongside development velocity so the team sees the full cost of AI-assisted delivery. A cheap-generated codebase that cannot be trusted is not actually cheap.

Key takeaways

• LLM-generated code can accelerate delivery, but it also raises the amount of verification required to trust the result.
• The strongest evidence in the article is not the code volume alone, but the combination of manual review and 2,059 total tests used to prove behavioural equivalence.
• Security-sensitive automation needs source-of-truth testing, clear simulation boundaries, and human ownership before it can be treated as reliable.

Key terms

• Behavioral Equivalence: Behavioral equivalence means two systems produce the same outcomes under the same inputs and conditions, even if their code looks different. In AI-assisted porting, this is the real test of correctness because surface similarity does not guarantee identical runtime behaviour or failure modes.
• Security Debt: Accumulated risk that builds when vulnerabilities, unsafe dependencies, and policy gaps are left unresolved across the software lifecycle. In AI-assisted development, security debt grows quickly because more code is produced, more decisions are made automatically, and remediation often lags behind delivery.
• Agent Harness: The agent harness is the runtime layer that wraps a model and turns it into an acting system. It usually includes the loop, tools, context handling, permissions, hooks, and logs. In security terms, it is often the real place where privilege sits and where identity evidence must be governed.

What's in the full article

ngrok's full post covers the implementation detail this analysis intentionally leaves at the framework level:

• The browser-based Kubernetes API design and how the simulated cluster maps to real kubelet, scheduler, and networking behaviour
• The testing harness that runs the same checks against k3s and webernetes to prove behavioural parity
• The detailed examples of LLM mistakes during porting, including shortcuts, invented helpers, and omitted tests
• The project graphs showing code growth, token usage, and cost over time

👉 ngrok's full post shows the porting workflow, test strategy, and token-cost breakdown behind webernetes.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management for practitioners building and reviewing operational controls. It helps security teams apply disciplined assurance when automation touches access, lifecycle, and trust decisions.