AI-generated code is outpacing testing capacity in the fourth wave

At a glance

What this is: Arxan Technologies argues that AI coding has shifted enterprise software risk from code production to test execution, triage, and quality validation at scale.

Why it matters: This matters to IAM, PAM, and security practitioners because faster software delivery increases the pressure on access, release, and compliance controls that must still govern humans and automation alike.

By the numbers:

• In many enterprise environments, 40–50% of code is now AI-generated, according to Arxan Technologies.
• Depending on industry and system complexity, 50–70% of testing effort is still manual, according to Arxan Technologies.
• Industry estimates suggest that 40–50% of automated test failures are not actual defects, according to Arxan Technologies.
• Studies and industry benchmarks indicate that up to 50% of QA and engineering time in automated environments is spent triaging failures, according to Arxan Technologies.

👉 Read Arxan Technologies' analysis of AI-generated code and the testing bottleneck

Context

AI-generated code changes the software delivery equation because code creation is no longer the slowest part of the pipeline. In the fourth wave described by Arxan Technologies, enterprises are producing more code faster, but testing, environment stability, and failure analysis have not scaled at the same pace.

That creates a governance problem as much as an engineering problem. When release velocity rises while validation remains manual or brittle, organisations increase the risk of defective code, delayed releases, and control gaps across the systems that depend on trustworthy software delivery, including identity and access workflows.

Key questions

Q: How should security teams control AI-assisted development when release speed keeps increasing?

A: Security teams should place policy and testing controls earlier in the delivery flow, then focus remediation on the systems where code velocity, privileged automation, and external dependencies intersect. The goal is not to slow every team equally. It is to prevent fast-moving pipelines from turning known weaknesses into repeated exposure.

Q: Why does AI-generated code create more operational risk even when testing is automated?

A: Because automation can still be brittle, stale, or poorly aligned with the codebase. AI increases the volume and pace of change, which raises the odds that test scripts, fixtures, and environments fall out of sync. That turns automation into a source of noise unless teams continuously govern drift.

Q: What are the signs that automated testing is failing as a control?

A: Look for high false-failure rates, long triage cycles, frequent rollbacks, and growing gaps between code change speed and validation speed. If teams spend more time proving whether a test failure is real than fixing the underlying issue, the control is no longer providing dependable assurance.

Q: Should organisations compare autonomous testing with manual QA, or use both?

A: Use both, but with clear boundaries. Manual QA still has value for judgement-heavy cases, while autonomous testing is better for scale, repetition, and environment coverage. The deciding factor is whether the team needs human interpretation or continuous execution across many variants.

Technical breakdown

Why AI-generated code increases testing entropy

AI-generated code raises testing entropy because change arrives faster than the supporting quality controls can adapt. Test suites, fixtures, and expected outcomes become misaligned as code paths evolve more frequently, which increases false failures and masks real defects. The problem is not simply volume. It is that the rate of change outpaces the feedback loop that keeps tests meaningful. In large environments, this creates a compounding reliability issue across CI/CD, staging, and production-like systems.

Practical implication: teams need test governance that tracks code change rate, not just test count.

Why triage becomes the new bottleneck

Once tests are written, the hard part becomes interpreting failure. A test can fail because the application is broken, because the environment changed, because the data is wrong, or because the script itself is stale. As automation scales, these causes overlap more often, and engineers spend more time distinguishing signal from noise. Agentic root cause analysis is attractive here because it can correlate failures across pipelines, environments, and dependencies, but it still depends on high-quality observability and stable data.

Practical implication: reduce triage load by linking test results to environment, data, and deployment context.

How autonomous testing changes release governance

Autonomous testing shifts quality from a checkpoint to a continuous control plane. Instead of treating tests as static artefacts, organisations need systems that heal brittle scripts, prioritise risk-based execution, and align validation with production-relevant conditions. This is especially important in mobile, web, and distributed application environments where device fragmentation, network variance, and third-party dependencies make deterministic testing difficult. The architectural challenge is orchestration, not just automation.

Practical implication: design testing as an adaptive control layer tied to release risk.

NHI Mgmt Group analysis

Test governance is now a release-security control, not a QA afterthought. When AI-generated code becomes a large share of enterprise output, validation quality directly affects the reliability of downstream security and identity workflows. Security teams should treat testing governance as part of change control, because weak validation increases the likelihood that access, release, and configuration defects move into production.

The new failure mode is not lack of automation, but automation drift. The article describes a world where test volume grows while test meaning decays. That pattern is familiar in identity governance too, where controls fail when lifecycle state, entitlements, and execution context diverge. The same governance lesson applies here: automation without alignment creates noise rather than assurance.

Adaptive triage is the named operational gap: enterprises are spending too much time proving whether a failure is real, environmental, or data-driven. That is a control problem, not just an engineering inconvenience. The practitioner conclusion is that quality programmes must prioritise failure classification and context correlation before scaling test volume further.

AI-assisted development magnifies the compliance value of runtime evidence. When release cycles compress, static approval alone becomes less persuasive than evidence that testing, verification, and exception handling are operating continuously. This is relevant to regulated environments where change evidence must be auditable and repeatable. The practical conclusion is to make validation telemetry part of governance reporting.

Identity and access controls still matter inside the delivery pipeline. As software delivery becomes more agentic, the question is who and what can change code, trigger tests, approve releases, and access data used in validation. That intersection between human identity, automation identity, and pipeline privilege is where modern release risk concentrates. Practitioners should govern those identities with the same seriousness as production access.

What this signals

Quality assurance is becoming a governance discipline. As AI-assisted development compresses build cycles, organisations need stronger evidence that testing, triage, and release approval are still trustworthy. The practical shift is toward observability-rich pipelines, where validation is treated as a measurable control rather than a delivery ritual.

Pipeline identity is part of software risk. The more code generation and test orchestration are delegated to automation, the more important it becomes to govern the identities that can change code, invoke tools, and approve releases. That makes privileged access review relevant not only in production, but also inside the delivery chain.

Test drift is the new hidden cost of speed. Enterprises that scale AI-generated code without adaptive validation will accumulate brittle checks, false failures, and slower incident response. The programme response is to treat test context, not just test volume, as the quality metric that matters.

For practitioners

• Map test ownership to release risk Tie test suites to the applications, environments, and data they protect so failures can be prioritised by business impact rather than by raw volume.
• Classify failures by root cause class Separate application defects, environment instability, stale scripts, and bad test data in the first triage pass to reduce noise and speed remediation.
• Instrument the delivery pipeline for context Capture deployment metadata, environment state, and data lineage alongside test output so teams can distinguish signal from drift faster.
• Govern pipeline identities and permissions Review which human users, automation accounts, and agentic systems can modify code, launch tests, or approve releases, and remove unnecessary standing access.

Key takeaways

• AI-generated code is shifting the main software constraint from creation to validation, and testing systems are not keeping up.
• The scale problem is measurable: large shares of testing remain manual, brittle, and time-consuming to triage.
• Security and identity teams should govern the delivery pipeline as a privileged environment, because release quality now affects operational trust.

Key terms

• Test Entropy: The growing mismatch between test assets and the systems they are supposed to validate. In fast-moving delivery environments, entropy rises when code, data, environment, and expected outcomes change faster than the test suite can adapt, producing noisy failures and weaker assurance.
• Failure Triage: The process of determining why a test or control failed and whether the failure reflects a real defect, an environment issue, stale automation, or bad data. At scale, triage becomes a governance problem because slow or inaccurate classification delays remediation and obscures risk.
• Adaptive Testing: A testing approach that adjusts execution, prioritisation, or maintenance based on application change and operational context. It is designed for environments where static scripts break quickly, and where validation must stay aligned with release velocity, infrastructure variability, and production-like conditions.
• Pipeline identity: A pipeline identity is the non-human identity a CI/CD workflow uses to authenticate to cloud, source control, secrets systems, and deployment targets. These identities are often overprivileged because they must automate multiple steps. That makes them high-value targets and a central concern in supply chain security.

What's in the full article

Arxan Technologies' full blog covers the operational detail this post intentionally leaves for the source:

• Benchmarks on how AI-generated code changes test volume, failure rates, and release cadence in enterprise environments.
• Operational discussion of self-healing test patterns, orchestration, and agentic root cause analysis across delivery pipelines.
• Context on how teams can reduce triage time when device fragmentation, environment instability, and data issues overlap.
• The vendor's detailed framing of autonomous testing platforms and where they sit in the software delivery lifecycle.

👉 The full Arxan Technologies post covers the test triage problem, autonomous testing shift, and release implications in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management for practitioners building modern control environments. It helps security and identity teams apply governance principles to systems where automation and human access overlap.