Topic starter
21/08/2026 4:54 pm
TL;DR: Prompt learning improved Claude Code by 5.19% on a by-repo split and 10.87% in-repo, using only system-prompt optimisation, LLM evals, and iterative feedback rather than model retraining or tool changes, according to Arize. The result matters because prompt governance is becoming a real control surface for AI coding agents, not just an implementation detail.
Editorial analysis by NHI Mgmt Group, based on content published by Arize: “CLAUDE.md Best Practices for Claude Code”.
Key questions
Q: How should security teams govern prompt changes in AI agent systems?
A: Treat prompt updates as production changes that can alter access, not just behaviour.
Q: Why do repository-specific instructions change AI agent behaviour?
A: Repository-specific instructions give the agent local context about file structure, coding conventions, testing habits, and accepted workflows.
Q: What are the signs that prompt optimisation is failing for a coding agent?
A: Common signs include patches that pass isolated tests but break repository conventions, repeated mistakes on the same API patterns, and output that improves in one codebase while degrading in another.
Practitioner guidance
- Establish prompt change control for coding agents Version system prompts, CLAUDE.md files, and any append-system-prompt rules in source control with reviewer approval before deployment.
- Separate general and repository-specific instructions Maintain a baseline instruction set for all repositories and a tightly scoped repo-specific layer so local conventions do not leak across projects.
- Require evaluator rationales for prompt updates Store LLM feedback explaining why outputs passed or failed, not just the final score, so prompt changes can be audited and rolled back.
Bottom line: Prompt learning can improve coding-agent output without touching model weights, which makes the instruction layer a real governance control.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
21/08/2026 5:11 pm
Prompt governance is becoming a first-class control surface for AI coding agents. This article shows that meaningful performance changes can come from system-prompt optimisation alone, without retraining or tool changes. For security teams, that means prompt content, prompt versioning, and prompt approval now belong in the same governance conversation as API permissions and repository access. The practitioner conclusion is simple: treat prompts as policy.
A few things that frame the scale:
- Claude Code-assisted commits leaked secrets at a rate of 3.2%, more than double the human-only baseline of 1.5%, with peaks reaching 31 secrets per 1,000 commits in August 2025, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: Should organisations prioritise prompt governance before model fine-tuning for coding agents?
A: Yes, when the issue is repository fit, instruction quality, or policy adherence. Prompt governance is cheaper, faster, and easier to audit than model retraining, and it often delivers measurable gains before any tuning work starts. Model changes make sense later, but only after teams have exhausted instruction-level controls and evaluation discipline.
👉 Read our full editorial: Prompt learning for Claude Code shows prompt-level gains