Join our Newsletter — 33% off our NHI Course
Home› Glossary› AI Security› Protected Classes
AI Security

Protected Classes

← Back to Glossary
By NHI Mgmt Group Updated October 31, 2026 Domain: AI Security

Protected classes are groups covered by anti-discrimination rules or fairness obligations, such as race, gender, age, or ethnicity. In AI governance, teams test model outcomes across these groups and their intersections to identify whether a system is producing uneven or unjust results in practice.

What Protected Classes Mean in AI Governance

Protected classes are not just demographic labels. In governance and fairness work, they define the populations that matter when a team checks whether a system produces systematically different outcomes for people covered by anti-discrimination obligations. The practical question is whether the model’s behaviour creates uneven treatment that could become unlawful, unfair, or difficult to justify.

Because protected classes are usually defined by law, policy, or organisational fairness standards, the exact list can vary by jurisdiction and use case. That makes the term operational as well as legal: teams need a shared understanding of which groups are in scope before they can measure disparity or interpret results consistently.

Why Intersections Matter

Checking only one protected class at a time can hide issues that emerge when groups overlap, such as age and gender together, or ethnicity and disability together. Intersectional analysis helps reveal compound effects that single-axis testing can miss, especially when the model’s decision rules behave differently across combinations of attributes rather than across one attribute alone.

This matters because a system can appear acceptable overall while still producing harmful or unstable outcomes for smaller subgroups. In practice, intersections are a way to test whether fairness claims survive more realistic population slices, rather than only broad averages.

How Organisations Use Protected-Class Testing

Teams usually use protected classes during model evaluation, bias review, and post-deployment monitoring. They compare outcomes across groups to look for gaps in error rates, approval rates, ranking positions, or other decision outputs that affect access, opportunity, or treatment. The goal is not to force identical outcomes in every case, but to surface patterns that warrant explanation or remediation.

That review often sits alongside data governance and policy review, because the organisation needs to know which attributes are being used directly, which are inferred, and which are only available for auditing. In regulated or high-impact settings, those choices shape what can be measured, how the evidence is interpreted, and who is accountable for the result.

What Makes Protected-Class Findings Hard to Interpret

Protected-class analysis can be misleading if the sample is too small, the data is incomplete, or the operational context changes between training and production. A difference in outcomes is a signal, not a conclusion. Teams still need to decide whether the gap reflects model design, skewed data, proxy variables, threshold choices, or a legitimate business rule that must be documented and defended.

For that reason, protected-class testing works best when it is tied to the actual decision process, not treated as a one-time checklist item. The same model can look fair in one workflow and problematic in another if the surrounding policy, human review, or eligibility logic changes.

Risk and Threat Considerations

Protected-class analysis exists because discriminatory or unequal outcomes can create legal, ethical, operational, and reputational exposure. In AI systems, the main risk is not only overt bias but also proxy effects, intersectional harm, and false confidence from aggregate metrics that hide subgroup failures.

Failure mechanism: A model may encode historical bias, learn proxies for protected attributes, or perform unevenly on underrepresented groups, causing biased decisions to persist at scale.

Impact: The organisation can face discriminatory outcomes, complaints, remediation work, regulatory scrutiny, and loss of trust in the system and the decision process around it.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

NIST AI RMF and NIST AI 600-1 set the technical controls, while EU AI Act, GDPR and ISO/IEC 42001:2023 define the regulatory obligations.

FrameworkControl / ReferenceRelevance
EU AI ActFairness and Non-Discrimination RequirementsGoverns AI systems where protected-class impacts and fairness obligations must be assessed.
Recommendation — Assess group impacts and document fairness controls for high-risk AI systems.
NIST AI RMFGOVERN — GovernFrames accountable AI governance, including fairness and harm management for impacted groups.
Recommendation — Assign accountable owners for fairness testing and escalation decisions.
NIST AI 600-1Map — Measure and ManageSupports measuring model impacts across affected populations and managing fairness risk.
Recommendation — Measure subgroup performance and manage identified disparities through controlled remediation.
GDPRData protection by design and DPIA obligationsApplies when personal data processing and automated decision impacts on individuals are in scope.
Recommendation — Build privacy and impact review into processing that can affect protected groups.
ISO/IEC 42001:2023AI management system requirementsSupports organisational AI governance, accountability, and risk treatment around fairness issues.
Recommendation — Embed fairness oversight into the AI management system and accountability model.

Practitioner Guidance

Governance implication: Define protected-class scope before testing so reviewers know which groups, intersections, and outcome measures are part of the fairness review. Keep the definition aligned with the applicable legal, policy, or programme context, then document how the review uses those categories in practice.

What to watch for: Treat protected-class testing as evidence of disparity, not proof of cause. When a gap appears, separate model behaviour, data quality, proxy features, and downstream human decisioning before deciding whether the issue is a tuning problem, a policy problem, or a broader governance failure.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Reviewed and updated by the NHIMG editorial team on October 31, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org