TL;DR: Tenzai tested three Base44 app specs across five models and found 254 vulnerabilities across 15 applications, with Base1 the only model producing no high or critical findings and no access-control vulnerabilities. Model choice changes the security baseline, but it still does not replace application testing or governance.
At a glance
What this is: This is Tenzai’s comparison of five app-building models on the same Base44 prompts, showing that model choice materially changed vulnerability volume, severity, and access-control failure modes.
Why it matters: IAM, app security, and AI governance teams should treat model selection as a security decision, because the resulting application can inherit different authorization and data exposure risks even when the platform and prompts stay constant.
By the numbers:
- Tenzai uncovered 254 vulnerabilities across the 15 applications.
Context
App builders that wrap an LLM around a prompt-to-code workflow do not produce neutral output. The model you choose shapes the security properties of the finished application, even when the builder, prompt, and deployment path stay the same.
That matters for identity and access governance because application flaws often become authorization flaws, data exposure paths, or trust boundary failures. In this case, the question was not whether the platform could generate software, but whether different models would consistently preserve control expectations across identical builds.
Key questions
A: Treat the model as a security-relevant component of the build pipeline, not just a productivity choice. Evaluate each model against the same application requirements, then require independent verification of the generated code before release. The key question is whether the application preserves authentication, authorisation, and input handling when the model changes.
Q: How should teams secure AI-generated applications before they reach production?
A: Treat security as a precondition of generation, not a review step at the end. Start with approved templates, shared authentication libraries, runtime secret retrieval, CI gates, and default-deny network policy. The goal is to make the safe path the easiest path, so generated code inherits controls instead of inventing them under pressure.
Q: Where do AI deployments most often fail in practice?
A: They fail when teams treat AI as isolated innovation instead of production infrastructure. The most common breakdowns are vulnerable packages left unpatched, credentials stored in insecure locations, and permissive defaults such as exposed services and missing encryption.
Q: Should teams compare different models before standardising on one for app generation?
A: Yes. The article shows that different models can produce materially different security outcomes even under identical conditions. Comparing models helps teams avoid standardising on one that systematically introduces higher-severity access-control or injection flaws, and it gives security teams evidence for model approval decisions.
Technical breakdown
How model choice changes application attack surface
When an app builder swaps only the underlying model, the surrounding orchestration stays constant while the generated code changes. That means the security outcome is not just a function of prompt quality or platform guardrails. Different models may interpret requirements with different assumptions about authentication, object access, input handling, and server-side enforcement. The result is a portfolio of applications that all appear functionally similar but diverge in their failure modes. In practice, that makes model selection part of the software threat model, not just a productivity choice. It also means a safe-looking builder can still emit risky code if the model produces weak control logic. See the Web Application Security Testing Guide.
Practical implication: Treat model selection as a security control input and test each model output as a distinct risk profile.
Why access-control bugs dominate AI-generated app risk
Access-control failures are especially dangerous in AI-generated applications because the code often looks complete while omitting the server-side checks that actually enforce who can see or change data. The article’s finding that every model except Base1 produced at least one critical or high flaw enabling unauthenticated access shows that abstraction layers do not guarantee authorisation correctness. In security terms, the builder can generate screens, routes, and workflows without reliably generating the policy enforcement behind them. That is a classic broken access control pattern, but now the source is model-generated code rather than hand-written development mistakes. See the Broken Access Control Prevention Guide.
Practical implication: Review every generated data path for server-side authorisation checks before the application is exposed.
Why injection findings still matter when the builder is 'AI-assisted'
Injection risk does not disappear because code was generated by a model. If the output fails to sanitise inputs, validate query construction, or constrain dynamic data access, the application still inherits classic injection exposure. The article shows that some models produced no high or critical injection findings, while others introduced high-severity NoSQL injection vulnerabilities. That difference matters because it demonstrates that model choice can shift the exact exploit path, not just the number of flaws. Security teams should therefore assess generated code for the same runtime attack classes they would expect in conventional development, including NoSQL-specific query abuse and data-layer injection.
Practical implication: Test model-generated code for injection paths with the same scrutiny used for manually written applications.
Threat narrative
Attacker objective: The attacker seeks to exploit weakly generated application logic to reach sensitive data or abuse exposed data paths.
- Entry occurs through an app builder workflow that turns prompts into deployable code without guaranteeing secure control implementation.
- Credential and authorisation abuse follow when generated applications expose data paths that permit unauthenticated access or weak server-side checks.
- Impact is realised when the resulting application leaks sensitive data or exposes runtime attack surfaces such as NoSQL injection.
Breaches seen in the wild
- United Nations breach 2021: Sakura Samurai used exposed Git credentials to reach 100,000+ UNEP staff records, then reported the flaw through the UN disclosure programme.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Model selection has become a security control, not a developer preference. When the same builder and prompts yield materially different vulnerability profiles, the organisation is effectively choosing a control posture every time it chooses a model. That shifts AI governance from abstract policy discussion to concrete application risk management, especially where generated code handles identity assertions, session state, or data access. Practitioners should treat model approval as part of software assurance, not an isolated AI procurement decision.
Access control remains the most fragile layer in generated applications. The most striking result is not just the vulnerability count, but that one model avoided high and critical access-control flaws while the others did not. That pattern suggests the governance problem is less about whether an app builder can produce code and more about whether the generated code preserves server-side authorisation discipline. In IAM terms, the application can only enforce least privilege if the generated logic actually implements it.
AI app builders create governance debt unless the output is independently verified. The article shows that stronger model output reduces risk, but does not eliminate it. That is the key operational lesson for teams building agentic or AI-assisted software: trust in the model cannot substitute for validation of the application boundary, especially where broken access control would expose data or functions. Practitioners should assume every generated build needs its own assurance path.
Runtime attack surfaces differ by model, which complicates assurance baselines. The article’s injection results show that model choice does not just change quality, it changes the class of defects that appear in deployment. That makes standard testing matrices insufficient if they assume a single development pattern across all generated outputs. Teams should build assurance processes that can compare models, not merely scan applications after deployment, because the risk is upstream of the code review stage.
What this signals
AI-assisted software development needs a per-model assurance model: the security posture of the generated application can vary enough that a single approval process is no longer sufficient. Organisations should expect to evaluate model output the way they evaluate third-party code, because the defect profile is now upstream of deployment and can differ by backend choice.
The governance challenge is not simply to detect flaws after generation. It is to define which models are acceptable for which classes of application logic, then verify that authentication, authorisation, and data-access controls survive the translation from prompt to code.
For practitioners
- Audit each model as a distinct security source Compare vulnerability profiles across models before you standardise on one for app generation, because identical prompts can still produce very different access-control and injection outcomes.
- Require server-side authorisation checks on every generated data path Verify that the application enforces access rules in backend logic, not only in UI flow or prompt instructions, especially for endpoints that return sensitive data.
- Run security testing on the generated application, not the model output Probe the deployed app for unauthenticated access, broken object handling, and injection exposure before any production release. See the Web Application Security Testing Guide.
- Separate model selection from release approval Treat the choice of model as one input to assurance, then require independent review of the resulting application before it is accepted into a production environment.
Key takeaways
- The article shows that AI app builders can produce materially different security outcomes even when the prompt and platform are unchanged.
- Access-control failures were the clearest governance problem, with one model standing out by avoiding high and critical findings in that area.
- Security teams should validate generated applications independently, because stronger model output lowers risk but does not eliminate insecure code paths.
Related NHI Mgmt Group resources: Broken Access Control Prevention Guide · Web Application Security Testing Guide · Authorisation Models Guide
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, OWASP API Security Top 10 and MITRE ATT&CK address the attack and risk surface, while OWASP ASVS and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Model-generated apps can mis-handle identity and privilege boundaries in deployed logic. |
| Recommendation — Review generated code for privilege boundaries and reject builds that weaken server-side authorization. | ||
| OWASP API Security Top 10 | API2 — Broken Authentication | The article's unauthenticated access findings map directly to API authentication failure. |
| Recommendation — Test every generated API for authentication bypass and deny release until auth controls hold. | ||
| OWASP ASVS | V8 — Authorization | Broken access control is the clearest defect class in the findings. |
| Recommendation — Apply authorization verification to all generated endpoints and enforce backend checks before launch. | ||
| MITRE ATT&CK | TA0006;TA0008 — Credential Access; Lateral Movement | The article highlights access paths that expose sensitive data and broaden attacker reach. |
| Recommendation — Map exposed data paths to credential abuse and lateral movement scenarios in threat testing. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | Choosing a model is a governance decision that affects downstream software risk. |
| Recommendation — Put model approval inside your AI governance process and require evidence-based acceptance criteria. | ||
Key terms
- AI App Builder: An AI app builder is a development environment that turns prompts into software with help from one or more models. Its security depends on both the platform guardrails and the model’s tendency to generate correct authentication, authorisation, and input-handling logic.
- Model-Dependent Security Variance: Model-dependent security variance is the difference in vulnerability profile that appears when the same application is generated with different underlying models. It matters because the security properties of the final code are not uniform, even if the prompt, platform, and deployment context are kept constant.
- Broken Access Control: Broken access control occurs when a system fails to restrict what an authenticated user, service, or workload can do. The issue often appears as missing checks, inconsistent enforcement, or excessive permissions. It is a structural weakness because attacks exploit the gap between verified identity and permitted action.
- AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.
Deepen your knowledge
NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is designed for practitioners who need to connect identity governance to software assurance and AI-driven change.
Published by the NHIMG editorial team on October 22, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org