Audit Evidence and Continuous Controls Monitoring Guide

Most failed audits are not caused by missing controls but by evidence that cannot prove the control worked: an undated screenshot, a user list nobody can show was complete, or a sign-off with no trace of what was reviewed. Continuous controls monitoring promises a steady stream of evidence, but a stream of weak evidence is still weak. This guide covers evidence quality and testing method: sufficiency and appropriateness, tests of design and operating effectiveness, sampling, control owner attestation, PBC lists, and keeping evidence fresh and traceable. It draws on PCAOB auditing standards, NIST SP 800-53A and SP 800-137, the AICPA Trust Services Criteria and ISO/IEC 27001:2022, and is written for control owners, GRC and internal audit teams, and engineers who produce evidence for SOX, SOC 2, ISO/IEC 27001 or NIST assessments.

Key takeaways

• Evidence is judged on quantity and quality. PCAOB AS 1105 calls these sufficiency and appropriateness; system-generated, independently sourced originals beat screenshots and verbal assurance.
• Design and operation are separate questions. A walkthrough shows a control could work; only testing across the period shows it did.
• Every report used as evidence needs its own proof. Populations and system reports must be shown complete and accurate before anything sampled from them counts.
• Freshness and lineage make continuous evidence usable. Each artefact should carry its source, query, timestamp and collector identity, and cover the period under test.
• Attestation is the weakest evidence. The SEC notes that self-assessments by the people operating a control generally provide less evidence; pair them with system data.

What makes audit evidence good enough

PCAOB AS 1105 sets the vocabulary most assurance work uses. "Sufficiency is the measure of the quantity of audit evidence", and "Appropriateness is the measure of the quality of audit evidence", meaning relevance and reliability. Evidence from a knowledgeable source independent of the company is more reliable, and "Evidence provided by original documents is more reliable than evidence provided by photocopies or facsimiles". Company-produced electronic information is more reliable when the controls over it are effective, which is why IT general controls sit underneath almost every other test. SOC 2 auditors, ISO/IEC 27001 certification bodies and NIST assessors apply the same logic. Structured audit evidence, meaning machine-readable records with defined fields rather than images, scores well because it can be re-queried and re-performed.

External audit evidence, obtained from outside the organisation or directly by the auditor, outweighs anything the control owner prepares, so design collection so auditors can pull or re-run evidence themselves.

Test of design versus test of operating effectiveness

AS 2201 has auditors test design by determining whether controls, operated as prescribed by people with the necessary authority and competence, would meet their objectives: "Walkthroughs that include these procedures ordinarily are sufficient to evaluate design effectiveness." Operating effectiveness is tested "by determining whether the control is operating as designed" and whether the performer has the authority and competence. Appendix A separates a deficiency in design (a necessary control is missing, or would not meet its objective even if operated as designed) from a deficiency in operation (a properly designed control does not operate as designed).

Assess design first. A quarterly access review that checks role names but not the entitlements behind them will pass every operating test and still miss the risk. Precision belongs in the design test: what exactly the reviewer looks at, what triggers follow-up, and what happens to exceptions.

Key control testing concentrates effort on the controls whose failure would leave a risk unaddressed. Internal control testing by management or internal audit should use the same method as the external auditor: AS 2201 lets auditors use others' work after assessing competence and objectivity ("The higher the degree of competence and objectivity, the greater use the auditor may make of the work"), so consistent test scripts directly reduce external audit hours.

Testing methods, sampling and cadence

AS 1105 lists inspection, observation, inquiry, confirmation, recalculation, reperformance and analytical procedures. NIST SP 800-53A Rev. 5 groups these into examine, interview and test, applied to specifications, mechanisms, activities and individuals, with depth and coverage each set at basic, focused or comprehensive. Each assessment objective contains determination statements, the testable units a good evidence request should map to. AS 2201 says reperformance gives the most evidence and inquiry the least, inquiry alone is never enough, and as control risk increases "the evidence that the auditor should obtain also increases".

Audit sampling

AS 1105 offers three selection methods: all items (100 percent examination), specific items with a defined characteristic, or audit sampling, applying a procedure "to less than 100 percent of the items". AS 2315 sets sample size from the tolerable rate of deviation, the likely rate of deviations, and the allowable risk of assessing control risk too low, and items "should be selected in such a way that the sample can be expected to be representative of the population", ideally across the whole period. Two details matter for evidence owners:

• Missing evidence is a deviation: AS 2315 says selected items the auditor cannot examine should ordinarily be treated as deviations. A lost ticket counts against you.
• Small samples tolerate almost nothing: its worked example uses a 5 percent tolerable rate, where zero deviations in 60 items supports reliance and two or more may not.

Control testing cadence should follow risk, not habit. NIST SP 800-137 treats "continuous" as assessed "at a frequency sufficient to support risk-based security decisions", with frequency set by factors including control volatility, system impact level, controls with identified weaknesses, threat and vulnerability information, and reporting requirements. Volatile, high-impact or recently failed controls deserve monthly or continuous testing; stable policy controls may need annual review.

Evidence freshness, lineage and drift

AS 2201 notes that "testing performed closer to the date of management's assessment provides more evidence than testing performed earlier", and interim testing needs further evidence for the remaining period. Audit evidence freshness therefore has two parts: when evidence was produced, and which period it covers. A January configuration export says nothing about September.

Audit evidence lineage is the ability to trace an artefact to the system and query that produced it and show it is unaltered. AS 1105 requires auditors using company-produced information to "Test the accuracy and completeness of the information", or the controls over it, and to judge whether it is "sufficiently precise and detailed". For external information the company supplies electronically, paragraph .10A adds testing "whether it has been modified by the company." Keep with every artefact:

• Source: system, tenant and environment (production, not a test copy).
• Query and parameters: the exact report, API call or SQL, with filters and date range.
• Run time: from a synchronised clock, with time zone.
• Collector identity: the person or read-only service account that pulled it.
• Completeness check: record count reconciled to source or a control total.
• Integrity: a hash and write-once storage.

Audit evidence drift is the gap that opens between what evidence says and what is now true: a control description that no longer matches the process, a collector that silently changed scope after an API change, or a population that excludes a newly acquired system. Compare record counts and population definitions between runs, re-walk key controls when systems change, and alert when a collector fails. A zero-row result is a finding, not a pass.

Continuous controls monitoring as an evidence layer

Continuous controls monitoring runs automated tests against live systems and records pass, fail and exception states over time. Tooling and test catalogues are covered in the Compliance Automation Guide; the question here is whether the output stands up as evidence. An audit evidence layer is the store between source systems and the audit, holding each result with its lineage, linked to a control and retained for the period. Audit evidence orchestration is the scheduling and routing of audit evidence collection so each artefact reaches the right request without manual copying.

Continuous control monitoring is management's activity: owners and second line watching their own controls. Continuous auditing is internal audit's: independent, frequent testing it relies on for its own conclusions. Data can be shared, but internal audit should own or validate any test logic it relies on; the IIA's Global Internal Audit Standards (five domains, 15 principles) set its independence and objectivity expectations.

Continuous audit evidence works where control state is machine-observable. Per action audit evidence, a record for every approval, deployment or privileged session, allows 100 percent testing and removes sampling for that control. SP 800-137 is a brake on overreach: "many aspects of ISCM programs are not easily automated", and controls must still be assessed "even if the monitoring of such controls cannot be automated". Management review controls and judgement-based approvals remain manual control testing. A green dashboard is management's assertion, not the auditor's conclusion: the auditor will still test design, population completeness and the ITGCs over the monitoring tool, including who can change a test's logic.

Control owner attestation

Attestation asks an owner to confirm that a control operated. It is cheap and scales, which is why it is overused. SEC interpretive guidance on management's ICFR evaluation (2007) states that "management must maintain reasonable support for its assessment", that self-assessments by the people who operate the control generally provide less evidence because the evaluator is less objective, and that where risk is high evidence "would ordinarily consist of direct testing".

• Make attestations specific: name the population, period, exceptions found and actions taken, not a tick box.
• Attach the data: reference the system extract the owner reviewed, with its lineage.
• Limit reliance: for key controls, attestation supplements testing; it never replaces it.
• Test attestations: second line or internal audit reperforms a sample each cycle.

PBC lists, evidence request lists and audit-ready packaging

A provided by client list (PBC list), or audit evidence request list, is the auditor's schedule of what must be supplied and by when. Most audit delays come from items that arrive late, incomplete or in a rejected form. A pre audit evidence package answers predictable requests before they are sent.

Audit evidence packaging means one folder or record per request, an index, file names carrying request and sample IDs, lineage metadata beside each file, and nothing outside scope. Over-supplying invites new questions and can expose personal data. Agree format, access route (read-only auditor access beats email) and retention at planning.

Where identity fits: access control testing

Access controls are the largest block of tests in most SOX, SOC 2 and ISO/IEC 27001 audits, and identity systems are the best source of identity based audit evidence because every account, group change, approval and sign-in is already recorded. Access audit evidence for the usual tests:

• Provisioning: new accounts in the period matched to approved requests; approval after the access date is a deviation.
• Deprovisioning: HR leavers reconciled to disable dates in every in-scope system, including local application accounts outside single sign-on.
• Access reviews: the exact entitlement listing the reviewer saw, with completeness proof, the decisions, and proof revocations happened. See the Access Reviews Guide.
• Non-human identities: service accounts, API keys and integration credentials are often missing from tested populations. Include them with owner and last use, and treat the collector's own credentials as privileged.

ERP control testing adds role assignments, sensitive transaction access and segregation of duties conflicts, tested from the application's own security tables rather than the directory. See the Segregation of Duties Guide.

How to implement

1. Inventory key controls: objective, owner, frequency, manual or automated, and the determination statements each evidences.
2. Write an evidence specification per control: population source, query, completeness check, artefacts per occurrence, retention and collector.
3. Agree it with the auditor before the period: format, sampling and reliance on automated collection. Changing evidence mid-period creates gaps.
4. Pilot five to ten controls: high-volume, machine-observable ones such as leaver deprovisioning, MFA coverage and change approval, collected automatically in parallel with existing manual evidence for one period.
5. Test the evidence: a second-line reviewer reperforms a sample of automated output. Fix lineage gaps before scaling.
6. Extend, keeping a rollback: keep each manual procedure documented until the auditor has accepted a full period of automated evidence.
7. Build the pre-audit package from last year's PBC list before fieldwork.

Choosing an approach

• Small organisation, first SOC 2 or ISO/IEC 27001: a shared evidence store with a fixed naming convention, system exports instead of screenshots, a PBC tracker, and automation for two or three high-volume access controls.
• Mid-size, several frameworks: a common control set with one evidence specification per control, collected once and mapped to each framework. See the Control Mapping Guide.
• Large or SOX-regulated: an evidence layer with lineage and integrity controls, internal audit validation of test logic, and ITGCs over the monitoring platform, which the auditor will test before relying on it.
• Automate or sample? Automate when control state is in a system, the population is large and the control runs often; keep manual testing for judgement controls and infrequent reviews.

Detection and measurement

Monitor the evidence pipeline like production: collector failures, zero-row or sharply changed record counts, permission errors on collector accounts, and logged changes to test logic or population definitions.

• First-time evidence acceptance rate: PBC items accepted without a query.
• Lineage-complete key controls: percentage whose evidence carries source, query, timestamp, collector and completeness check.
• Median evidence age at request: days between collection and request, against the period tested.
• Deviations found by management first: control failures detected by monitoring before the auditor found them.
• Reperformance agreement: sampled automated results a reviewer independently reproduces.

Framework mapping

Common failure patterns

• Screenshots by default: one point in time, often undated, impossible to re-perform. Use them only where no export exists.
• Incomplete populations: the leaver test covers the directory but not systems with local accounts. The Ledger Connect Kit compromise involved a former employee's unrevoked npm access, exactly what a directory-only population misses.
• Evidence a task was raised, not done: rotation or revocation tickets closed without proof of outcome. The Trivy supply chain attack turned on an incomplete credential rotation.
• Design never retested: operating tests pass for years on a control whose scope no longer matches the system after a migration.
• Monitoring presented as audit: a dashboard offered as the conclusion, with no ITGCs over the tool and no proof the population is complete.
• Attestation in place of testing: quarterly "I confirm" sign-offs on key controls with no system evidence.

Practitioner checklist

• Write an evidence specification for every key control: population, query, completeness check, artefacts, retention.
• Test design before operation, and retest design after system changes or migrations.
• Prove completeness and accuracy of every report or population used as evidence.
• Store evidence with source, query, timestamp, collector identity and hash, in write-once storage.
• Agree evidence formats, sampling and reliance on automation with the auditor before the period.
• Treat missing evidence for a sampled item as a deviation, and design collection to prevent it.
• Use attestation only as a supplement for key controls, and reperform a sample each cycle.
• Include non-human identities and local accounts in every access control population.
• Alert on collector failures, zero-row results and changes to test logic.

Standards and references

• PCAOB: AS 1105, Audit Evidence
• PCAOB: AS 2201, An Audit of Internal Control Over Financial Reporting That Is Integrated with An Audit of Financial Statements
• PCAOB: AS 2315, Audit Sampling
• NIST: SP 800-53A Rev. 5, Assessing Security and Privacy Controls in Information Systems and Organizations
• NIST: SP 800-137, Information Security Continuous Monitoring (ISCM) for Federal Information Systems and Organizations
• SEC: Commission Guidance Regarding Management's Report on Internal Control Over Financial Reporting
• AICPA and CIMA: 2017 Trust Services Criteria (With Revised Points of Focus, 2022)
• ISO: ISO/IEC 27001:2022 Information security management systems, Requirements

Related NHI Mgmt Group resources: SOX IT General Controls (ITGC) Guide · SOC 2 Guide · ISO 27001 Certification Guide · GRC Platform Buyer's Guide