Security teams should treat agentic AppSec as a guided find, fix, verify workflow. Point the AI coding agent at a running application, run a full scan, inspect the evidence behind each finding, remediate in source, and rescan to confirm the issue is closed. The value comes from making runtime testing part of normal build and release practice.
Why This Matters for Security Teams
Agentic application security testing in live development environments matters because the application is no longer a static target. The coding agent can generate code, invoke tools, open files, and chain actions across the workspace, so the testing process has to validate both the software and the behaviour around it. Guidance from the OWASP Agentic AI Top 10 and NHI-focused research such as OWASP Agentic Applications Top 10 both point to the same problem: runtime behaviour creates findings that purely static scans miss.
This is especially important for secrets exposure, unsafe tool use, and broken guardrails. The NHIMG report The State of Secrets in AppSec notes that the average estimated time to remediate a leaked secret is 27 days, which shows how quickly a small testing miss can become an operational exposure. In live development, the goal is not just to detect issues, but to prove they are fixed in the running environment before they reach release.
In practice, many security teams discover agentic app risks only after the coding agent has already exposed a token, widened access, or modified the wrong control path.
How It Works in Practice
The most effective workflow is guided find, fix, verify. Point the agent at a running application, let it execute realistic tests, inspect the evidence behind each finding, then remediate in source and rescan to confirm the issue is gone. This keeps AppSec tied to the actual build, runtime configuration, and current secrets state instead of a stale snapshot.
For agentic systems, the scan should test more than code quality. It should challenge prompt handling, tool permissions, credential storage, data exposure, and whether the agent can be induced to perform an unintended action. The NIST AI Risk Management Framework is useful here because it pushes teams to assess risk continuously rather than treat AI behaviour as a one-time review. Likewise, CSA MAESTRO agentic AI threat modeling framework helps structure the test around the agent, its tools, and the trust boundaries between them.
- Run the agent against a live dev instance with test accounts and seeded data.
- Capture exact prompts, tool calls, file edits, API requests, and secret access events.
- Validate each finding against source code, config, and runtime evidence before fixing.
- Rescan after each remediation to confirm the issue is actually closed.
- Document any residual risk where the agent still has broad execution authority.
Where possible, pair runtime testing with identity-aware controls such as ephemeral credentials, workload-scoped tokens, and policy checks at request time. That approach aligns well with the operational lessons in Analysis of Claude Code Security and the broader agentic risk patterns documented by OWASP NHI Top 10. These controls tend to break down when the development environment is shared, long-lived, or full of unmanaged side effects because the scanner can no longer distinguish intentional test behaviour from real production-like action.
Common Variations and Edge Cases
Tighter runtime testing often increases coordination overhead, requiring organisations to balance developer velocity against the risk of testing in an environment that can mutate state. That tradeoff is real, especially when agentic tools can create records, call external APIs, or trigger workflow automation during a scan.
Current guidance suggests using constrained test tenants, disposable credentials, and clearly labeled data fixtures when the application has destructive side effects. For multi-agent pipelines, test each hop separately as well as the end-to-end flow, because one agent may be secure while the downstream tool chain is not. There is no universal standard for this yet, but the emerging best practice is to combine threat modeling with live verification rather than rely on either one alone.
Teams should also watch for environments where scans are noisy or incomplete because of throttling, cache layers, or asynchronous jobs. In those cases, a clean rescan does not automatically mean the issue is fixed if the vulnerable path was never exercised. The most reliable programs supplement the live scan with evidence review, change tracking, and a narrow set of regression tests for the highest-risk agent actions. The NHIMG incident coverage in Amazon Q AI Coding Agent Compromised shows why even trusted coding agents need active verification, not just policy statements.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, CSA MAESTRO and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A2 | Agent tool abuse and unsafe actions are central to live testing. |
| CSA MAESTRO | T1 | MAESTRO frames agent, tool, and trust-boundary testing in runtime. |
| NIST AI RMF | AI RMF supports continuous risk evaluation during agentic testing. | |
| OWASP Non-Human Identity Top 10 | NHI-03 | Runtime tests often expose leaked or overlong-lived secrets. |
| NIST CSF 2.0 | PR.DS-1 | Protecting data in transit and at rest matters during live testing. |
Treat live agentic scans as continuous risk checks, not one-time validation.
Related resources from NHI Mgmt Group
- How should security teams govern agentic IDEs in development environments?
- How should security teams manage application risk in fast-moving development environments?
- How should security teams decide where to run dynamic application testing?
- How should security teams adapt dynamic application security testing for API driven and web 3.0 environments?
