Join our Newsletter — 33% off our NHI Course
Home› Guides› Privacy in Security Logging and Monitoring Guide: Lawful,…
Guide Privacy & Data Protection

Privacy in Security Logging and Monitoring Guide: Lawful, Minimised and Still Useful SIEM Data

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 27 October 2026 15 min read
On this page

A SIEM is one of the largest stores of personal data in most organisations, yet it is rarely designed, documented or protected as one. Authentication logs, proxy and DNS records, email gateway events, endpoint telemetry and cloud audit trails all identify people: usernames, IP addresses, device IDs, browsing history, message metadata and sometimes content. Data protection law does not stop security monitoring, and the GDPR explicitly recognises network and information security as a legitimate interest, but it does require a lawful basis, minimisation, defined retention, protection and transparency. This guide shows how to build security logging that meets those duties without losing detection value: what personal data sits in each log source, how to minimise and pseudonymise at ingestion, how to set retention, who may re-identify, and how to handle subject access and suppliers. It is for SOC leads, detection engineers, SIEM owners, DPOs and privacy engineers. Workforce monitoring beyond security logging, such as productivity tools and covert monitoring, is covered in the Employee Monitoring and Privacy Guide.

Key takeaways

  • Security logging is lawful processing, not an exemption. Legitimate interests (GDPR Article 6(1)(f), with Recital 49 on network and information security) usually covers it, but only to the extent strictly necessary and proportionate, and only with a documented assessment.
  • Minimise at the source, not in the SIEM. Drop payloads, bodies and fields with no detection use before ingestion; what never lands cannot leak, be requested or need deletion.
  • Pseudonymise analyst views, keep a controlled path back. Keyed pseudonyms for user identifiers preserve correlation, and a logged, approved re-identification step keeps investigations possible.
  • Retention is a decision per log class. Hot, warm and cold tiers with different access rules, aligned to threat dwell time and legal requirements, beat a single "keep for seven years" setting.
  • Logs leak too. Secrets, tokens and personal data in logs are a breach waiting to happen, as the exposed DeepSeek log database showed, so protect log stores like production data.

Under the GDPR and UK GDPR, Recital 49 states that processing personal data to the extent strictly necessary and proportionate for ensuring network and information security is a legitimate interest of the controller. That supports Article 6(1)(f) as the usual lawful basis, alongside legal obligations in some sectors (for example security and incident reporting duties under NIS2 or financial regulation). It does not remove the other principles.

  • Legitimate interests assessment: the ICO's monitoring workers guidance recommends the three-part test of purpose, necessity and balancing, documented in a legitimate interests assessment, and notes that workers can object to processing under this basis. Write one assessment per monitoring capability (EDR, proxy, email security, UEBA), not one for "security".
  • Data minimisation and storage limitation: GDPR Article 5(1)(c) and 5(1)(e) apply to logs as much as to customer data. The ICO says personal information obtained from monitoring workers must not be kept for longer than necessary.
  • DPIA: the ICO says a DPIA must be carried out before processing likely to result in high risk, and gives keystroke monitoring as an example. UEBA, DLP with content inspection, TLS inspection and full packet capture usually meet that threshold. See the DPIA Guide.
  • Transparency: workers must be told about monitoring in an accessible way; covert monitoring is exceptional, authorised by senior management, time-limited and covered by a DPIA. Customers and visitors whose IP addresses and sessions are logged need coverage in the privacy notice.
  • Special category data: monitoring can capture sensitive data incidentally, such as emails to a trade union representative or occupational health. Where that is likely, the ICO says you must identify a processing condition.
  • Status note: the ICO's monitoring workers guidance is under review following the Data (Use and Access) Act and may change.

What personal data sits in each log source

Log sourcePersonal data typically presentDetection valueMinimisation options
Identity provider and authenticationUsername, UPN, IP address, device, location, user agent, MFA methodVery high: account takeover, impossible travel, MFA fatigueKeep; pseudonymise usernames in analyst views; coarse location after the hot tier
Web proxy and secure web gatewayFull URLs including query strings, user, IP, categoriesHigh: command and control, phishing clicks, exfiltrationStrip query strings by default; keep domain and path; category only for sensitive categories (health, religion)
DNS resolver logsQueried names, client IP or hostHigh: C2 domains, tunnellingKeep names; map client to host, not person, where possible
Email gatewaySender, recipients, subject, attachments, sometimes bodyHigh: phishing and malware deliveryKeep headers and verdicts; do not ingest bodies; hash attachments
Endpoint detection and responseProcess command lines, file paths with names, user, network connectionsVery highKeep; mask file names under personal folders after the hot tier
Cloud and SaaS audit logsActor, IP, object names, sometimes content previewsVery high: data access, privilege changesKeep events; drop content previews and message excerpts
Application logsCustomer IDs, emails, request bodies, session tokensMedium to high: fraud, abuse, API attacksNever log tokens or passwords; log customer IDs not emails; drop bodies
DLP and content inspectionMatched content snippets, file names, recipientsHigh for insider and exfiltration casesStore match type and count, not the matched content, unless an incident is opened
Full packet captureEverything, including contentHigh for short-term forensicsShort rolling window only; OMB M-21-31 sets 72 hours for US federal agencies

IP addresses, device IDs and usernames are personal data where the organisation can link them to a person, which a security team almost always can. Machine and service account events are not personal data in themselves, but they often carry the name of the human who owns or used the account.

Minimising at ingestion

The OWASP Logging Cheat Sheet lists data that should usually not be recorded directly in logs, and should instead be removed, masked, sanitised, hashed or encrypted: session identifiers, access tokens, authentication passwords, database connection strings, encryption keys and other primary secrets, payment and bank data, sensitive personal data such as health and government identifiers, data above the logging system's classification, and data a user has opted out of or not consented to.

  • Filter in the pipeline: apply drop and mask rules in the log shipper or pipeline (for example the collection agent, a stream processor or the SIEM's ingest pipeline) before data is indexed. Rules at the search layer leave the raw data on disk.
  • Use schemas: normalise into a schema such as OCSF, ECS or ASIM, and maintain an allow list of fields per source. New fields default to dropped until reviewed.
  • Secret scanning on logs: run detectors for API keys, tokens, JWTs and passwords on log streams and alert the owning team when one appears; the fix belongs in the application, not only in the filter.
  • Prevent log injection: structured formats such as JSON, input length limits and output encoding, as OWASP recommends, also stop personal data being smuggled into fields that bypass masking.
  • Cost alignment: minimisation reduces ingest licences and storage, which is often the argument that gets it funded.

Pseudonymised analysis with controlled re-identification

Most detection logic needs to know that events belong to the same user, not who the user is. ENISA's 2021 report on advanced pseudonymisation includes cybersecurity use cases such as telemetry and reputation systems, and the EDPB's draft Guidelines 01/2025 describe keeping pseudonymisation secrets separate, access-controlled, logged and rate-limited.

  1. Tokenise identifiers at ingestion: replace usernames, email addresses and employee IDs with a keyed pseudonym (HMAC with a key held in a KMS), consistently across sources so correlation still works.
  2. Keep enrichment separate: department, role and privilege level can be added as attributes without the name, which keeps UEBA peer grouping working.
  3. Re-identify on a case, not on a whim: analysts request identity resolution against an open incident ticket; the request, approver and result are logged; senior approval is needed for re-identifying people in HR-sensitive contexts.
  4. Exempt where justified: privileged and administrative accounts, service accounts and accounts already linked to an open incident can stay in clear for speed.
  5. Test: pilot with one data source and one use case (for example impossible travel) and confirm detection rates and analyst time are unchanged before rolling out.

Pseudonymised logs are still personal data for the organisation. The gain is in reduced exposure to analysts, suppliers and anyone who steals the SIEM data. For method selection and key handling, see the Pseudonymisation Techniques Guide.

Retention and tiering

TierTypical contentTypical periodAccess
Hot (searchable)Normalised events used for detection and triage30 to 90 daysSOC analysts, detection engineers
WarmReduced fields for threat hunting and investigationsUp to 12 monthsSenior analysts and incident responders on request
Cold archiveCompressed raw or reduced logs for forensics and legal needs12 to 30 months, or as required by lawIncident response with approval; immutable storage
Short windowFull packet capture, content inspection snippetsHours to daysNamed forensic staff
Legal holdRecords tied to an incident, claim or investigationUntil released by legalIncident and legal teams
  • Justify periods with dwell time and duties: intrusions are often found weeks or months after they start, which justifies months, not days, for authentication and network metadata. Document the reasoning in the log retention schedule.
  • Know external minimums: US federal agencies must keep most log categories for 12 months in active storage and 18 months in cold storage under OMB M-21-31, with full packet capture at 72 hours; sector rules (PCI DSS, financial regulators) also set minimums. These are justification for the period, not permission to keep everything.
  • Delete for real: enforce retention with index lifecycle policies and storage lifecycle rules, and include backups and exported copies.

See the Log Management and Retention Guide for platform-level retention design.

Access, suppliers and transfers

  • Least privilege in the SIEM: role-based access with data-level restrictions, so HR investigation data, executive mailboxes and DLP content are visible only to named staff. Log and review all searches in the SIEM itself, which is what OWASP means by recording and monitoring all log access.
  • Protect log integrity and confidentiality: immutable or write-once storage for archives, encryption at rest and in transit, and separation between the SOC's log access and the administrators whose actions are being logged.
  • MSSPs and cloud SIEMs are processors: they need an Article 28 contract that covers purpose limits, sub-processors, location, retention, deletion at contract end and support for subject requests. The ICO notes that using a third-party tool does not transfer responsibility. See the Cybersecurity for MSPs and MSSPs Guide.
  • International transfers: follow-the-sun SOCs and offshore analysts are transfers of personal data. Cover them with the appropriate transfer mechanism and a transfer risk assessment. See the International Data Transfers Map.
  • AI in the SOC: sending log extracts to an LLM service is another disclosure. Apply the same minimisation and pseudonymisation to prompts, and check the provider's retention and training terms. See the Agentic AI in the SOC Guide.

Subject access and other rights

  • Logs are in scope: the ICO says monitoring data about workers must be made available to them on request, subject to exemptions. Plan how to search the SIEM by person, including through pseudonyms.
  • Use exemptions narrowly: where disclosure would prejudice an investigation or reveal detection methods, apply the relevant exemption case by case and record why, rather than refusing all log data as a category.
  • Objections: under legitimate interests individuals can object, but compelling legitimate grounds such as protecting systems and other users will usually override an objection to core security logging. Record the assessment.
  • Erasure: retention limits are the main erasure mechanism for logs; deleting individual entries on request can damage evidential integrity. Explain this in the privacy notice. See the DSAR Guide: Handling Data Subject Requests.

Where identity fits

Identity data is both the most valuable and most sensitive part of security logs. Two practical points matter. First, map non-human identities (service accounts, workload identities, API clients) to owning teams rather than named people in the logs, which preserves accountability without adding personal data. Second, the SIEM's own ingestion connectors, API keys and service principals have read access to almost everything; manage them as privileged NHIs with rotation, scoping and monitoring. See the Cloud Logging and Audit Trail Guide.

How to implement: a phased approach

  1. Inventory log sources: for every source feeding the SIEM, data lake or MSSP, list the fields, the personal data categories, the detection use cases that need them and the current retention.
  2. Document the lawful basis: write legitimate interests assessments per capability and DPIAs for high-risk ones (UEBA, DLP, TLS inspection, packet capture), and update privacy notices for staff and customers.
  3. Cut fields with no use case: drop or mask fields that no detection rule, hunt or investigation playbook uses. Run in parallel for two weeks and compare alert volumes before switching off the old feed.
  4. Introduce pseudonymisation for one source: start with proxy or SaaS logs, prove detection is unchanged, then extend.
  5. Set and enforce retention tiers: configure lifecycle policies per log class and verify deletion, including in backups.
  6. Lock down access: role-based and data-level access in the SIEM, search auditing, and review of who can re-identify.
  7. Contract and transfers: review MSSP and SIEM vendor contracts and transfer mechanisms.
  8. Roll back safely: keep previous pipeline configurations versioned so a masking rule that breaks a detection can be reverted quickly, and fix the detection rather than removing the control.

Detection and measurement

MetricWhat it showsTarget
Log sources with a documented field inventory, use case and retentionAccountability coverage100% of SIEM sources
Secrets or tokens detected in log streams per monthApplication logging hygieneFalling towards zero, each fixed at source
Share of user identifiers pseudonymised in analyst viewsExposure reductionRising to all non-privileged users
Re-identification requests without a linked incidentMisuse of the re-identification pathZero
Data held beyond the retention scheduleStorage limitation in practiceZero, verified quarterly
Mean time to answer a subject access request covering logsRights handling readinessWithin the statutory period

Framework mapping

FrameworkReferenceRelevance
GDPR and UK GDPRArticles 5(1)(c), 5(1)(e), 6(1)(f), 15, 28, 35; Recital 49Minimisation, retention, lawful basis, access rights, processors, DPIA
NIST SP 800-53 Rev. 5AU-3(3) Limit Personally Identifiable Information Elements; AU-9; AU-11Limiting PII in audit records, protecting and retaining them
NIST SP 800-53 Rev. 5PT-2, PT-3, SI-12, AC-6Authority and purpose for processing, retention, least privilege on log data
NIST CSF 2.0PR.PS-04, DE.CM-01, DE.CM-03, GV.OC-03Generating logs, monitoring networks and personnel activity, legal and privacy requirements
ISO/IEC 27001:2022Annex A 8.15 Logging; 8.16 Monitoring activities; 5.34 Privacy and protection of PIILogging and monitoring controls that respect PII obligations
CIS Controls v8.1Control 8 Audit Log ManagementLog collection, retention and review. See the CIS Control 8 Audit Log Management Guide.

Common failure patterns

  • Logging everything because storage is cheap: full URLs, request bodies and email content ingested by default, with no use case, creating a store nobody can justify to a regulator.
  • Secrets and personal data in application logs: tokens, passwords and chat content written to debug logs and then exposed. In the DeepSeek database exposure, an unauthenticated ClickHouse log table held over a million entries with plaintext chat history and API keys.
  • One retention period for all logs: seven years for everything, or the SIEM default, with no reasoning recorded and no deletion from backups.
  • Unrestricted analyst search: every analyst can search any person's browsing and email metadata, with no audit of searches.
  • Forgotten processors: MSSPs, cloud SIEMs and AI assistants receiving logs without processor terms or transfer assessments.
  • Refusing all access requests for logs: treating security logs as automatically exempt instead of assessing each request.

Practitioner checklist

  • Inventory personal data fields in every log source and link each to a detection or investigation use case.
  • Write a legitimate interests assessment per monitoring capability and DPIAs for high-risk ones.
  • Drop or mask fields without a use case in the pipeline, before indexing.
  • Block tokens, passwords and secrets from logs at the application, and scan log streams for them.
  • Pseudonymise user identifiers in analyst views with keyed pseudonyms and a logged re-identification workflow.
  • Set retention tiers per log class, document the reasons, and verify deletion including backups.
  • Apply role-based and data-level access in the SIEM and audit every search.
  • Put Article 28 terms and transfer assessments in place for MSSPs, cloud SIEMs and AI services.
  • Prepare a tested process to answer subject access requests that include log data.

Standards and references

Related NHI Mgmt Group resources: Employee Monitoring and Privacy Guide · Log Management and Retention Guide · Pseudonymisation Techniques Guide · GDPR Compliance Guide for Security Teams

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 27 October 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.