Trust and Safety Programme Guide: DSA and OSA Ready

Every platform that lets users post, sell, message or advertise will be abused, and the EU Digital Services Act and the UK Online Safety Act now turn the way a platform handles that abuse into an auditable legal duty. A trust and safety (T&S) programme is the operating model that prevents, detects and remediates abuse: scams, fake accounts, illegal content, harassment, counterfeit goods and fraudulent ads. This guide covers how to structure the function, write enforceable policy, choose enforcement tooling, run notice and action and appeals, publish transparency data, and meet DSA and OSA duties. It is written for heads of trust and safety, fraud, risk and security leaders, and product executives at marketplaces, social, gaming, dating and community platforms. Day-to-day queue management is covered in the Trust and Safety Operations guide; this page is about building the programme.

Key takeaways

Trust and safety is a system, not a moderation queue: policy, detection, enforcement, appeals, data, law enforcement liaison and safety by design have to work together, as the Trust & Safety Professional Association (TSPA) curriculum lays out.
Regulation makes the process auditable: the DSA requires notice and action, statements of reasons, internal complaints and transparency reporting; the OSA requires illegal content risk assessments and safety duties enforced by Ofcom since 17 March 2025.
Fraud and scams are now a core T&S harm: the OSA puts specific fraudulent advertising duties on the largest services, and scam content drives most consumer harm on marketplaces and social platforms.
Automation needs human governance: classifiers and rules handle volume, but both laws expect platforms to explain proactive technology, allow appeals and keep error rates under control.
Enforcement tools are privileged access: the consoles that remove content, ban users and read private data are a target for bribery and account takeover, so treat them like any other privileged system.

What a trust and safety programme covers

TSPA's curriculum defines trust and safety as the work of preventing, addressing and remediating abusive behaviour on digital platforms, and organises the field into twelve areas. They make a sound blueprint for the programme's scope.

Programme area	What it delivers	Typical owner
Policy	Community guidelines, harm definitions, enforcement ladders, regional variations	T&S policy team with legal
Content moderation and operations	Review queues, quality assurance, moderator wellbeing, appeals handling	T&S operations, often with outsourced vendors
Automated systems and AI	Classifiers, hash matching, rules engines, risk scoring	T&S engineering and data science
Investigations and intelligence	Network takedowns, scam and fraud ring analysis, threat actor tracking	Investigations team, often shared with fraud
Law enforcement response	Lawful request handling, emergency disclosures, referrals	Legal and law enforcement response team
Data and measurement	Prevalence, enforcement accuracy, response time metrics	T&S analytics
Transparency reporting	Public reports on enforcement and governance metrics	Policy and analytics with legal sign-off
Legal and regulatory	DSA, OSA and other compliance, risk assessments, audits	Legal and compliance
Safety by design and privacy	Harm review of features before launch, privacy-preserving detection	Product, privacy and T&S jointly
External engagement	Trusted flaggers, researchers, industry signal sharing, civil society	Policy and partnerships

The overlap with fraud and security is large: fake account creation, account takeover, scam ads and seller fraud are handled by T&S on some platforms and by fraud teams on others. Decide who owns each harm explicitly. See the Fraud and Cyber Convergence Guide.

Operating model and team design

Accountable executive: one senior leader owns harm outcomes and regulatory compliance, reporting to the board on risk. Both the DSA and the Ofcom codes expect governance and accountability for these duties, and the largest DSA services need an independent compliance function.
Policy separated from enforcement: those who write rules should not be measured on enforcement volume, so policy stays principled and enforcement metrics stay honest.
Tiered review: automation and frontline reviewers handle clear cases; specialists handle high-severity harms (child safety, terrorism, credible threats, large fraud rings); escalation goes to policy and legal.
Vendors with oversight: outsourced moderation is common. Keep policy, quality calibration, appeals sampling and data access controls in-house.
Wellbeing: moderators exposed to graphic content need exposure limits, rotation, counselling and tooling that blurs or greyscales by default. This is a retention issue as well as a duty of care.

Small vs large platforms: a start-up can begin with one accountable lead, a clear policy, a reporting button routed to a shared queue, hash matching for known illegal material through industry programmes, and a simple appeals inbox. A platform near the DSA very large threshold (45 million average monthly active recipients in the EU, measured over six months) needs specialist teams by harm, systemic risk assessments, independent audits and a regulatory reporting pipeline.

Policy and enforcement design

Policy is only enforceable if reviewers and machines apply it consistently. TSPA's curriculum treats policy as covering how rules are developed, the abusive behaviours they target and the enforcement methods used.

Write for reviewers: public guidelines explain the rule; internal guidance gives definitions, examples, edge cases and decision trees so two reviewers reach the same outcome.
Enforcement ladder: define proportionate actions: label, reduce distribution, remove content, feature restriction, temporary suspension, permanent ban, referral to law enforcement. The DSA recitals expect suspension of repeat abusers after a prior warning, with redress.
Actor, behaviour and content: enforce on the account or network, not only the item. Scam operations recycle content across many accounts; banning posts without linking accounts lets them return. See the Fake Account Creation and Sign-up Fraud Guide and the Bot Farms and Fake Engagement Guide.
Terms of service alignment: OSA section 10 requires terms that explain how users are protected from illegal content, are applied consistently, and describe any proactive technology used, including its kind, when it is used and how it works.

Enforcement tooling and automation

Technique	Best for	Limits	Governance needed
Hash matching (perceptual and cryptographic)	Known illegal images and video, re-uploads of removed content	Only finds previously identified material	Trusted hash sources, false match review
Keyword and rules engines	Spam, scam phrases, prohibited goods listings, velocity limits	Easy to evade, brittle across languages	Rule change control, precision sampling
Machine learning classifiers	High-volume harms such as nudity, hate speech, scam ads	Bias and drift, opaque decisions	Evaluation sets, per-language accuracy, human review of edge cases
Large language model review	Nuanced policy reasoning at scale, triage, summarising reports	Hallucination, prompt injection from user content, cost	Policy-grounded prompts, audit logs, human sign-off on severe actions
Account and network signals	Fake accounts, coordinated inauthentic behaviour, fraud rings	Privacy constraints, collateral bans	Data minimisation, appeals, investigator review
User and trusted flagger reports	Context the platform cannot see, new harms	Abuse of reporting, backlog	Prioritisation, notice quality checks, misuse controls

The DSA recitals say voluntary moderation does not lose the liability exemptions where it is carried out in good faith and diligently, including keeping automated tools reliable enough to limit errors. That is a design requirement: measure classifier precision and recall per harm and language, and route low-confidence decisions to humans.

Securing the enforcement stack

Moderation consoles, user data lookups and ban tools are privileged systems. Insiders and outsourced agents with these tools are targets for bribery, and attackers who take over a reviewer account can unban accounts or harvest personal data.

Least privilege by queue: reviewers see only the data needed for the case in front of them; bulk export is disabled by default.
Strong authentication: phishing-resistant MFA for all T&S tools, including vendor staff.
Machine identities: classifiers, LLM review agents and automation pipelines that can take enforcement actions run under their own scoped service identities, with rate limits and logs. See the Non-Human Identity (NHI) Guide.
Insider monitoring: alert on unusual lookups, mass reversals and access to high-profile accounts. See the Insider Threat Programme Guide.

Notice and action, appeals and transparency under the DSA

The Digital Services Act (Regulation (EU) 2022/2065) scales duties by service type and size: basic duties for all intermediaries, more for hosting services, more again for online platforms, and the most for very large online platforms and search engines, which the European Commission supervises directly.

Duty	DSA provision	What the programme must provide
Notice and action	Art. 16	An easy, user-friendly mechanism for anyone to report illegal content, with notices precise and substantiated enough to act on
Statement of reasons	Art. 17	A clear explanation to the affected user of each restriction, its grounds and the redress available
Internal complaint handling	Art. 20	Appeals available for at least six months after the decision
Out-of-court dispute settlement	Art. 21	Good-faith engagement with certified dispute bodies (outcomes not binding)
Trusted flaggers	Art. 22	Priority processing of notices from designated trusted flaggers
Measures against misuse	Art. 23	Suspension of users who frequently post manifestly illegal content, or abuse reporting, after a warning
Transparency reporting	Arts. 15 and 24	Machine-readable reports on moderation; statements of reasons submitted to the Commission's public database
Marketplace traceability	Online marketplace duties	Seller verification and reasonable efforts at random product checks
Systemic risk (very large services)	Arts. 34, 35 and 37	Annual risk assessment, mitigation measures and independent audit

Micro and small enterprises are exempt from some duties unless designated very large. Fines for breaches can reach 6% of worldwide annual turnover. Security-relevant DSA duties are covered in the Digital Services Act Security Obligations Guide.

Online Safety Act duties in the UK

The Online Safety Act 2023 applies to user-to-user and search services with links to the UK, regulated by Ofcom. As checked on 8 November 2026:

Illegal content risk assessment: in-scope providers had to complete one by 16 March 2025, and keep it current.
Illegal content safety duties (section 10): enforceable from 17 March 2025, when Ofcom's illegal content codes (issued 24 February 2025, from its statement of 16 December 2024) took effect. Providers must use proportionate measures to prevent users encountering priority illegal content, minimise how long it is present, and take it down swiftly when aware. Ofcom lists over 130 priority offences, including fraud offences. Providers either follow the codes' measures or show alternative measures are effective.
2026 amendment: the Crime and Policing Act 2026 added a duty, from 29 June 2026, to take down reported intimate image content, and identical or substantially similar content, within 48 hours of a report.
Children's safety duties: services likely to be accessed by children had to complete children's risk assessments, with protection duties enforceable from summer 2025. See the Age Assurance Guide.
Fraudulent advertising (section 38): Category 1 services must have proportionate systems to prevent users encountering paid-for fraudulent ads, minimise the time they are present and remove them swiftly, and explain any proactive technology in their terms. Category 2A search services have parallel duties. These depend on Ofcom's categorisation register and codes, so confirm whether your service is categorised. See the UK Online Safety Act Fraud Duties Guide.
Penalties: Ofcom can fine up to £18 million or 10% of qualifying worldwide revenue, whichever is greater.

Because the DSA and OSA overlap heavily, build one notice, decision and appeals pipeline with jurisdiction tags, not two parallel processes.

Scam ads and fraud on platforms

Fraud is the harm where T&S, fraud and security most need to cooperate. Scam ads, fake investment endorsements, fake sellers and romance scam profiles are abuse of the platform and fraud against its users at once.

Advertiser verification: verify identity and, for financial services ads, authorisation status before ads run.
Pre-publication screening: classifiers and rules on creative, landing pages and destination domains, with re-screening when landing pages change after approval.
Fast reporting routes: a dedicated route for banks, regulators and trusted flaggers to report scam ads, with response time targets.
Network enforcement: link advertisers by payment instruments, devices and domains, and remove the network, not the single ad.
Signal sharing: exchange indicators with industry and financial institutions. See the Marketplace Seller and Platform Fraud Guide.

Building the programme: a phased approach

Assess: run a harm and legal applicability assessment: which DSA tier and OSA categories apply, which harms occur on the service, current volumes and gaps. Treat the OSA illegal content risk assessment as the backbone.
Govern: appoint the accountable executive, set policy ownership, define the enforcement ladder and approve the risk register.
Build the pipeline: reporting, triage, review, decision, statement of reasons, appeal and data capture as one case flow with audit logs.
Automate carefully: introduce classifiers in shadow mode first, measure precision against human decisions, then enable automated action for high-precision cases only; keep a kill switch to roll back to human review.
Measure and report: build the metrics below into dashboards that feed transparency reports directly, so published numbers come from the same data as operations.
Review: run safety by design reviews on new features, refresh risk assessments after significant changes, and run red team exercises against policy and tooling.

Detection and measurement

Prevalence: the share of views or items that violate policy, estimated by sampling, which shows exposure better than removal counts.
Time to action: median and 90th percentile time from report or detection to decision, split by severity.
Enforcement accuracy: precision of automated and human decisions from quality sampling, and appeal overturn rate.
Proactive detection rate: share of actioned content found before any user report.
Repeat offender rate: share of banned actors detected returning within 30 days.
Scam ad exposure: fraudulent ad impressions per million served and time to removal after a trusted report. See the Fraud Metrics and KPIs Guide.

Framework mapping

Programme element	NIST CSF 2.0
Accountable executive and roles	GV.RR-01, GV.RR-02
Policy and terms of service	GV.PO-01
Legal and regulatory duties	GV.OC-03
Harm risk assessment	ID.RA-03, ID.RA-04
Access to enforcement tooling	PR.AA-05
Specialist reviewer training	PR.AT-02
Monitoring of reviewer and system activity	DE.CM-03
Programme review and improvement	GV.OV-03, ID.IM-03

Common failure patterns

Removing content, not actors: deleting scam posts one by one while the network behind them recreates accounts faster than reviewers can act.
Automation without measurement: enabling auto-removal without precision testing per language, producing wrongful takedowns, appeal backlogs and regulatory complaints.
Transparency numbers from a different system: published figures assembled by hand from spreadsheets that do not reconcile with operational data, which fails audit.
Two compliance pipelines: separate DSA and OSA processes with different policies, producing inconsistent decisions for the same content.
Over-privileged reviewer tools: every reviewer, including vendor staff, able to view any account's private data and reverse any ban, with no anomaly monitoring.
Safety reviewed after launch: new features such as direct messaging or live streaming shipped without a harm review, then retrofitted after abuse appears.

Practitioner checklist

Map which DSA tier and OSA categories apply to each service, and record the decision.
Appoint an accountable executive and separate policy ownership from enforcement metrics.
Publish clear terms and internal enforcement guidance, including how proactive technology is used.
Run one case pipeline from notice to decision, statement of reasons and appeal, with audit logs.
Test classifiers in shadow mode and track precision per harm and language before automating action.
Enforce against accounts and networks, linking signals across identities, devices and payments.
Protect enforcement tools with phishing-resistant MFA, least privilege and insider monitoring.
Keep the illegal content risk assessment current and review it after significant product changes.
Generate transparency reports from operational data and reconcile before publishing.

Trust and Safety Programme Guide: Building Platform Integrity Teams