Notifications
Clear all
Topic starter
19/08/2026 3:30 pm
TL;DR: LLMs can generate large codebases quickly, but correctness still depends on human review and a test harness that compares behaviour against a real Kubernetes cluster, according to ngrok. The bigger lesson is that speed without verification creates hidden quality debt, especially when AI is used to port complex systems.
NHIMG editorial — based on content published by ngrok: LLM-assisted porting of Kubernetes into the browser with webernetes
By the numbers:
- The final week of the project consumed $1,811.64 in API-equivalent token cost.
Questions worth separating out
Q: How should teams validate AI-generated mobile code before release?
A: Teams should validate AI-generated mobile code against mobile-specific security standards, then test authenticated flows that exercise token handling, local storage, and backend API access.
Q: Why do AI-generated ports need behavioural testing, not just code review?
A: Because code can look correct while behaving differently at runtime.
Q: What do teams get wrong about using LLMs for infrastructure or platform code?
A: They often assume the model’s speed reduces the need for assurance.
Practitioner guidance
- Require dual verification for AI-generated code Make human code review and automated behavioural testing mandatory for every generated change set that touches orchestration, policy, or access logic.
- Create source-of-truth comparison harnesses Run the same test cases against the generated system and the reference implementation so you can detect mismatches in state handling, lifecycle transitions, and API responses.
- Separate simulation from production controls Document which parts of an AI-built environment are educational or illustrative and which parts are operationally trusted.
What's in the full article
ngrok's full post covers the implementation detail this analysis intentionally leaves at the framework level:
- The browser-based Kubernetes API design and how the simulated cluster maps to real kubelet, scheduler, and networking behaviour
- The testing harness that runs the same checks against k3s and webernetes to prove behavioural parity
- The detailed examples of LLM mistakes during porting, including shortcuts, invented helpers, and omitted tests
- The project graphs showing code growth, token usage, and cost over time
👉 Read ngrok's retrospective on building webernetes with LLMs →
LLM-generated code at scale: what do review and tests actually prove?
Explore further
19/08/2026 4:13 pm
AI-generated code increases verification debt, not just delivery speed. The article shows that LLMs can accelerate production of large code volumes, but the real burden shifts to review, testing, and behavioural comparison. That pattern is familiar in identity and security engineering, where fast automation can conceal subtle defects in policy logic, privilege handling, or workflow orchestration. The practitioner conclusion is simple: if the system matters, verification is part of the build, not an optional follow-up.
A question worth separating out:
Q: Should organisations use AI-generated code in security-sensitive workflows?
A: Yes, but only with strict controls around review, testing, and ownership. For security-sensitive workflows, the key question is not whether AI can generate code, but whether the organisation can prove its behaviour, maintainability, and failure modes. Without that proof, the delivery gain is offset by hidden risk.
👉 Read our full editorial: LLM-assisted code generation still needs review and tests
