Use manual testing for scenarios that depend on human judgement, physical interaction, setup friction, and environmental variation. Those are the places where scripted checks usually miss what real users and real devices experience. The strongest manual coverage is exploratory, especially early in the lifecycle when behaviour is still changing and assumptions are not yet stable.
What belongs in manual testing for an IoT programme?
Manual testing is most valuable where the outcome depends on human interpretation or on conditions that are hard to script reliably. In IoT, that usually includes device onboarding, pairing, physical setup, UI flows, recovery steps, and behaviours that change with environment, interference, or latency. If the result varies by user behaviour or real-world context, a person should still validate it.
Teams should treat manual testing as a way to expose gaps between the lab and the field. That includes poor instructions, confusing device states, brittle resets, unexpected sensor behaviour, and failure modes that automated checks may not represent accurately. The goal is not to replace automation, but to reserve human effort for the parts of the system where observation, judgement, and improvisation matter most.
A useful rule is to ask whether the test depends on stable inputs and a repeatable path. If yes, automate it. If the test depends on subjective assessment, messy setup, inconsistent conditions, or interactions across hardware, mobile apps, and cloud services, keep it manual or at least start there before converting it into a scripted check.
Which IoT scenarios are strongest candidates for manual coverage?
The strongest manual candidates are exploratory sessions around first-time use, recovery, and edge conditions. That includes what happens when a device is misconfigured, restarted mid-setup, paired in the wrong order, moved between networks, or exposed to weak connectivity. These are often the moments when users get stuck and when product assumptions fail.
Manual testing is also useful for verifying whether a device behaves sensibly in the presence of physical variation. Lighting, distance, placement, noise, temperature, timing, and signal quality can all change the actual outcome. A script can confirm a sensor response; it cannot always tell you whether the response is understandable, usable, or safe in context.
Where IoT combines hardware, mobile, and backend services, manual checks help validate the full user journey. That is especially true for onboarding, credential entry, firmware updates, permissions prompts, local fallback behaviour, and error handling. Those flows often succeed technically while still being confusing or fragile from an operator’s point of view.
How should teams choose between manual and automated tests?
The best split is based on repeatability, risk, and change rate. Automate stable regression checks that need to run often and produce the same result every time. Keep manual coverage for the scenarios that are still evolving, where the test outcome is partly qualitative, or where the device or environment introduces too much variation for a dependable script.
Early in the lifecycle, manual testing should get more weight because requirements, interactions, and failure patterns are still moving. Once a behaviour stabilises, convert the check into automation if it has high repeat value and a clear pass or fail condition. That gives you fast feedback without losing coverage of the messy cases that only real usage reveals.
The decision is not only about engineering efficiency. Manual testing is often the only practical way to notice when a flow is technically correct but operationally poor, such as a pairing process that works only when the user performs steps in an exact order or a recovery procedure that succeeds but is too error-prone to trust under pressure.
Risk and Threat Considerations
IoT testing gaps often show up first as usability failures, then as security and reliability problems. If setup is confusing or recovery is brittle, users create workarounds, skip steps, or leave devices in unsafe intermediate states, which can widen exposure and make support or incident handling harder.
Failure mechanism: Automated checks miss context-sensitive behaviour, so the team believes the device is fit for use when real users, real networks, or real physical conditions produce different outcomes. That creates blind spots in onboarding, fallback behaviour, and error recovery.
Impact: Devices can ship with hidden operational weakness, inconsistent state transitions, or unsafe failure handling that only appears after deployment, when fixes are slower and the blast radius is larger.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP ASVS, CIS Controls v8 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP ASVS | V15 — Secure Coding and Architecture | Manual test selection follows system behavior and lifecycle maturity. |
| Recommendation — Use V15 to test evolving IoT flows before converting stable checks into automation. | ||
| CIS Controls v8 | CIS-8 — Audit Log Management | IoT validation must verify observable state and failure handling in practice. |
| Recommendation — Validate that key IoT events remain observable during setup, reset, and recovery. | ||
| NIST CSF 2.0 | DE.CM-01 — Monitoring for Anomalies and Events | Manual IoT testing should expose anomalies that scripted checks may miss. |
| Recommendation — Monitor field behaviour for anomalies that only appear under real environmental variation. | ||
Practitioner Guidance
What to prioritise: Start manual coverage with first-use journeys, recovery paths, and any flow where a user must interpret feedback from a device, app, and backend at the same time. Those are the places where the highest-value discovery usually happens.
What to verify: Confirm that a manual test is checking something a script cannot yet evaluate reliably, such as physical setup friction, ambiguous status signals, or behaviour under variable environmental conditions. If the outcome is objectively repeatable, it probably belongs in automation next.
Practitioner takeaway: Manual testing in IoT is most valuable when it reveals the gap between a controlled lab pass and a real-world user experience, especially for flows that depend on judgement, physical context, and imperfect conditions.
Related resources from NHI Mgmt Group
- How should teams decide whether CIAM belongs in the same IAM programme as workforce access?
- How do security teams decide when vulnerability scanning should feed a broader remediation programme?
- How do security and compliance teams decide which fraud signals matter most in an identity programme?
- How do security teams decide when DAST should be part of a unified AppSec programme?