Trust and Safety Career Guide: Roles and Routes

Trust and safety jobs are hard to read from the outside because the same work hides behind dozens of job titles, and many applicants do not realise how much of the role is investigation, policy judgement and data rather than content review. This guide explains the functions inside a trust and safety (T&S) team, the skills and tools each one uses, realistic routes in from security, fraud, policy, law, data and community work, the credentials that help, and how careers progress. It also covers wellbeing, because exposure to harmful material is part of many T&S roles and should shape which role you choose. It is written for job seekers and for security and fraud professionals considering a move. For how a programme is built and run, see the Trust and Safety Programme Guide; this page is about the people and the jobs.

Key takeaways

Trust and safety is a family of jobs, not one job. The Trust & Safety Professional Association (TSPA) describes 12 functions, from content policy and operations to engineering, legal and threat discovery.
Read responsibilities, not titles. TSPA advises searching adverts for words such as "safety", "policy", "compliance" and "moderation" because titles vary widely between companies.
Coding is optional for most roles. SQL or Python matters mainly for analyst and investigation roles; engineering and data science are the exceptions where code is core.
Regulation is creating demand. The EU Digital Services Act and the UK Online Safety Act turned notice handling, appeals, risk assessment and transparency reporting into legal duties with named deadlines.
Choose your exposure deliberately. Moderation, child safety and violent extremism work carry real psychological load; ask about exposure limits and support before you accept an offer.

What the role does

A T&S team decides what content and behaviour is acceptable on a platform, writes it down as policy, enforces it at scale and measures whether enforcement works. TSPA notes that the field has moved from reactive work (removing flagged content, answering legal requests) to proactive work, with teams now consulted during early product design to anticipate abuse. Smaller companies often combine several functions in one person.

The table summarises the functions TSPA lists and the typical titles you will see in adverts.

Function	What the work involves	Typical titles	Content exposure
Content policy	Writing the rules for allowed content, balancing values, user expectations, law and community input; briefing leadership	Content Policy Manager, Policy Analyst, Knowledge Management	Moderate: reviews examples when drafting and testing rules
Product policy	Product-specific rules (for example stricter rules for ads), advising product teams across jurisdictions	Product Policy Manager, Product Policy Associate	Low to moderate
Operations	Day-to-day moderation, quality assurance, training, workflow and capacity management, crisis response, vendor management	Specialist, Analyst, Investigator, Programme Manager, Vendor Manager	High for reviewers and QA; lower for programme roles
Threat discovery and research	Investigating networks of abuse and bad actors, sharing findings internally and with law enforcement	Abuse Investigator, Threat Intelligence Investigator	Moderate to high, depending on harm area
Law enforcement response	Assessing legal requests for legal and terms-of-service compliance, urgent escalations, people in crisis	Law Enforcement Response Analyst, Investigations Analyst, Incident Response Analyst	Moderate to high
Data science and analytics	Measuring prevalence of violations, evaluating detection, forecasting trends, countering adversarial behaviour	Data Analyst, Data Scientist, Data Engineer	Low, mostly aggregate data
Engineering	Machine learning models for enforcement, reporting and review tools, takedown systems	Software Engineer, Security Engineer	Low
Legal	Requests from regulators and law enforcement, legal risk, product advice	Counsel, privacy and copyright specialists	Low to moderate
Product management	Strategy for reducing violating content in a harm area or across QA and transparency reporting	Product Manager	Low
Content design and strategy	User education, help centre articles, in-product warnings and interventions	Content Designer, Content Strategist	Low
Public policy and communications	Relationships with regulators, governments and NGOs; regional policy advice	Public Policy Manager	Low

The exposure column is a guide to typical work, not a TSPA classification. Ask the hiring manager directly how much raw content a role reviews each week.

How T&S differs from security and fraud work

Harm, not just compromise: security protects systems and data; T&S protects users from each other and from abuse of legitimate features. Many harms involve no hacking at all.
Policy is the control: the main control is a written rule applied consistently, so judgement, documentation and appeals matter as much as detection.
Shared adversaries: spam, fake accounts, account takeover and scams sit across all three teams. See the Fake Account Creation and Sign-up Fraud Guide and the Bot Farms and Fake Engagement Guide.

Core skills and tools

TSPA lists the skills valued across roles as project management, cross-functional collaboration, clear communication, analytical thinking, self-direction, adaptability, humility and resilience. The table adds the role-specific skills and the kinds of tools you should expect to learn.

Skill area	What good looks like	Tools and evidence	Most needed in
Policy reasoning	Writing rules that reviewers in many countries apply the same way; spotting edge cases and over or under enforcement	Policy drafts, enforcement guidelines, decision trees, appeal outcomes	Content and product policy, operations
Investigation	Linking accounts and behaviour into networks, documenting findings to an evidential standard	Case management, link analysis, open-source research, platform signals (device, IP, account age)	Threat discovery, law enforcement response
Data analysis	Measuring prevalence, precision and recall of enforcement, appeal overturn rates	SQL, Python or R, dashboards, sampling and labelling methods	Analytics, operations, investigations
Automation and ML literacy	Understanding classifiers, thresholds, hash matching and human review queues; knowing when automation fails	Classifier metrics, labelling guidelines, review tooling	Engineering, product, operations
Regulatory knowledge	Translating DSA and Online Safety Act duties into processes and records	Statements of reasons, notice logs, risk assessments, transparency reports	Policy, legal, compliance
Privacy and data handling	Collecting only what enforcement needs; lawful disclosure to law enforcement	Data minimisation records, disclosure logs	Legal, law enforcement response, engineering
Crisis and incident handling	Running a surge (elections, attacks, viral harm) with clear roles and escalation	Incident playbooks, on-call rotas, post-incident reviews	Operations, policy, communications

Regulatory literacy is now a core skill. Under the EU Digital Services Act, platforms must offer a notice mechanism for illegal content, explain removals to users, provide internal appeals and access to out-of-court dispute settlement, prioritise notices from trusted flaggers and, for very large platforms with more than 45 million monthly EU users, assess and mitigate systemic risks. In the UK, Ofcom's roadmap required in-scope services to complete an illegal content risk assessment by 16 March 2025, with children's risk assessments and child safety duties following in 2025. Candidates who can explain how a notice becomes a statement of reasons and an auditable record stand out. See the Digital Services Act Security Obligations Guide and the UK Online Safety Act Fraud Duties Guide.

Routes into the role

TSPA states that no single degree is required and that T&S draws on human-computer interaction, social sciences, law, and privacy or security. It also says there is no set career path: large companies may have internal ladders, smaller ones usually do not. The table maps common starting points to realistic first roles.

Starting point	What transfers	Realistic first T&S role	Gap to close
SOC or threat intelligence analyst	Triage, investigation, adversary tracking, case notes	Abuse investigator, threat discovery analyst	Policy reasoning, harm areas beyond malware
Fraud or AML analyst	Scam typologies, mule networks, account signals, regulated decision records	Scams and fraud policy or investigations	Content harms, freedom of expression trade-offs
Customer support or community management	Platform knowledge, user empathy, escalation handling	Content review specialist, escalations analyst	Data skills (SQL), written policy analysis
Law, policing or public policy	Legal reasoning, evidence, regulator engagement	Law enforcement response, policy analyst, regulatory compliance	Platform operations and tooling
Data analyst or scientist	Measurement, experimentation, modelling	T&S data analyst, enforcement quality analyst	Labelling and policy nuance
Academic researcher	Research methods, subject expertise (extremism, child safety, misinformation)	Policy specialist, threat researcher, training lead	Translating research into business decisions
Software or security engineer	Building systems, abuse-resistant design	T&S engineer, integrity engineer	Policy context, review workflow design

Learn the vocabulary: work through the TSPA curriculum (policy, moderation and operations, transparency reporting, automated systems, law enforcement, investigations, safety by design, privacy) and read two or three platform transparency reports.
Pick a harm area: scams, child safety, extremism, harassment, spam and fake accounts, or misinformation. Specialists are easier to hire than generalists.
Build visible evidence: write a mock policy with enforcement guidelines and edge cases, or a short investigation write-up using only public data. Never collect or share harmful material to build a portfolio.
Use adjacent roles: TSPA suggests community management, content quality, internships, government, civil society and research institutes as ways in. Internal moves from support, fraud or security teams are often fastest.
Network deliberately: TSPA runs a job board, mentoring, regional summits and the annual TrustCon conference; membership is open to students and to civil society and academic professionals.
Prepare for scenario interviews: expect policy cases ("should this post stay up?"), prioritisation under surge, and a data or investigation exercise. Explain your reasoning and the trade-off, not just the answer.

Watch for recruitment fraud while you job hunt: fake recruiters target applicants with task scams and fake technical tests. See the Job and Task Scams Guide.

Wellbeing and choosing your exposure

TSPA's careers guidance says working on heavy topics can affect mental health and how you see the world, and names recurring pressures: accepting that no measure fully solves a problem, trading off over-enforcement against under-enforcement, and feeling under-appreciated because success means harm that never becomes visible. Treat wellbeing as a job criterion, not an afterthought.

Ask before you accept: how many hours a week of raw content review the role involves, whether exposure is rotated across harm areas, and whether reviewers can opt out of specific categories.
Ask about tooling: whether review tools blur, greyscale, mute or crop media by default, and whether hash matching removes known material before a human sees it.
Ask about support: access to specialist psychological support, how vendor moderators (often outsourced) receive the same support, and how leave works after a traumatic case.
Build your own routine: TSPA suggests deliberately noticing positive stories and good user behaviour to counter tunnel vision. Clear start and end rituals, peer debriefs and not reviewing content at home all help.

Many of the same practices apply to incident responders. See Incident Responder Wellbeing.

Certifications that help

There is no single industry-standard T&S certification. TSPA describes degrees and certifications as ways to build relevant expertise rather than entry tickets. Credentials from adjacent fields are useful when they match the harm area or function you target.

Credential	Issuer	Why it helps in T&S	Best fit
CIPP/E	IAPP	ANAB-accredited proof of European data protection knowledge, relevant to disclosure, data retention and DSA work	Legal, law enforcement response, policy
CFE (Certified Fraud Examiner)	ACFE	Fraud schemes, investigation and evidence, relevant to scams and marketplace abuse	Scams investigations, fraud policy
Entry security certification (for example CompTIA Security+ or ISC2 CC)	CompTIA, ISC2	Shared vocabulary with security teams on account takeover, bots and infrastructure	Integrity engineering, threat discovery
Threat intelligence or OSINT training	Various	Network analysis and attribution methods	Threat discovery and research

See the CIPP/E and CIPP/US Certification Guide, the CFE Certified Fraud Examiner Certification Guide and the Cybersecurity Certifications Guide. Check exam details on the issuer's own page before booking; they change.

Progression

TSPA lists several growth paths: specialising deeply in one harm area, broadening across domains, moving into technical or product roles, management, joining smaller organisations for wider scope, and mentoring or consulting. In practice the ladders look like this:

Track	Early career	Mid career	Senior
Operations	Content review specialist, escalations analyst	QA lead, operations programme manager, vendor manager	Head of T&S operations
Policy	Policy associate or analyst	Policy manager for a harm area or product	Head of policy, public policy director
Investigations	Investigations analyst, LER analyst	Senior abuse investigator, threat intelligence lead	Head of threat discovery or investigations
Data and engineering	Data analyst, T&S engineer	Data scientist, ML engineer for enforcement	Integrity engineering or data science lead
Compliance	Regulatory analyst	DSA or Online Safety compliance manager	Head of online safety compliance, accountable lead for a regulator

T&S skills also travel. Investigators move into threat intelligence and fraud; analysts into security data science; compliance staff into wider GRC. See the Threat Intelligence Analyst Career Guide, the Fraud Analyst career guide and the Security Data Scientist Career Guide. TSPA runs a compensation survey for the field; this guide does not quote pay figures because no official statistics series covers T&S as a distinct occupation.

Common failure patterns

Applying by title only: missing good roles called "integrity", "platform safety" or "community operations" because the word "trust" is absent.
Treating it as moderation only: candidates who talk only about removing bad posts and ignore measurement, appeals and regulation lose out to those who show the whole loop.
No harm-area focus: generalist applications compete with specialists who already know the typologies and the law in scams, child safety or extremism.
Ignoring exposure: accepting a high-exposure review role without asking about limits and support is the most common cause of early burnout.
Unsafe portfolio work: collecting or reposting harmful content "for research" can break law and platform terms; use public reports and synthetic examples.

Practitioner checklist

Read the TSPA functions and roles page and choose one function as your target.
Choose one harm area and learn its typologies, signals and relevant law.
Work through the TSPA curriculum chapters for your target function.
Learn enough SQL to pull and sample enforcement data if you target analyst or investigation roles.
Explain how DSA notice and action, statements of reasons and appeals work, and what Ofcom's risk assessment duties require.
Write one mock policy with edge cases and one public-data investigation write-up.
Ask every employer about content exposure, rotation, tooling and psychological support.
Add one credential that matches your harm area, such as CIPP/E for legal work or CFE for scams.

Standards and references

Related NHI Mgmt Group resources: Trust and Safety Programme Guide · Trust and Safety Operations · Cybersecurity Careers Guide · Fraud Analyst

Trust and Safety Career Guide: Roles, Skills, Entry Routes and Wellbeing