Once an intruder automates collection, data is gathered faster than any analyst can watch, and the speed itself becomes the best thing to detect. MITRE ATT&CK tracks this as T1119 Automated Collection, in the Collection tactic, across Windows, Linux, macOS, IaaS, SaaS and Office Suite platforms. Scripts that sweep file shares for document types, cloud API calls that page through storage or mailboxes, ETL pipelines repurposed to copy data out, and now AI agents tasked with gathering and sorting stolen data all fall under it. This guide focuses on what makes collection automated rather than manual: the rate, repetition and client signatures that give it away on endpoints and in cloud and SaaS logs, the logging you must turn on first, and how to respond. It is for detection engineers, SOC analysts and cloud security teams. For the tactic as a whole, see the ATT&CK Collection Tactic Guide.
Key takeaways
- Automation leaves a rhythm. Bursts of file reads by one process, fixed-interval collection runs and thousands of API calls from one identity are the common thread across T1119 procedures.
- Cloud and SaaS collection is API-driven. ATT&CK notes adversaries may use cloud APIs, data pipelines, command line interfaces or ETL services, so the evidence is in audit and API logs, not on endpoints.
- The logs that matter are often off by default. AWS CloudTrail does not log S3 object reads unless data events are enabled, and Microsoft Graph activity logs must be routed through diagnostic settings.
- Non-human identities do much of the collecting. Service principals, access keys and OAuth apps are ideal collectors because they rarely sign in interactively. Baseline them by identity.
- ATT&CK's mitigations are limited. It maps only M1029 Remote Data Storage and M1041 Encrypt Sensitive Information, and notes these may not stop a persistent intruder. Detection and least privilege carry the weight.
How the technique works
ATT&CK describes T1119 as an adversary, once established, using automated techniques to collect internal data, for example a command and scripting interpreter (T1059) searching for and copying files that match criteria such as type, location or name at set intervals. The function can also be built into remote access tools. ATT&CK links it to File and Directory Discovery (T1083), Lateral Tool Transfer (T1570), Cloud Service Dashboard (T1538) and Cloud Storage Object Discovery (T1619). What follows collection is usually staging and archiving; see the T1560 and T1074 Archive and Stage Collected Data Defensive Guide.
| Form | Typical procedure (from ATT&CK examples) | Where the evidence is |
|---|---|---|
| Extension sweeps on hosts | Malware or scripts that search drives, including removable media, for lists of document extensions and copy matches to a staging folder (for example Ramsay, Zebrocy, Patchwork tooling) | EDR process and file events, Sysmon Event ID 1 and 11, auditd |
| Scheduled or looped collection | Tools that collect a working directory every few hours, or monitor folders and collect changed files (for example LoFiSe, FunnyDream) | Scheduled task and service creation, periodic file access by one process |
| Directory and identity data | Bulk export of directory data, such as ROADTools gathering Azure AD data through the Graph API, or menuPass using Csvde against Active Directory | Directory audit logs, Graph activity logs, LDAP query logging |
| Mailbox and document APIs | HAFNIUM using Microsoft Graph to take data from email, OneDrive and SharePoint | Graph activity logs, Microsoft Purview audit (MailItemsAccessed) |
| Cloud configuration harvesting | Pacu automatically collecting CloudFormation templates, EC2 user data and IAM data | CloudTrail management and data events |
| Credential and secret harvesting | Supply chain malware such as Shai-Hulud and TeamPCP Cloud Stealer collecting secrets from many file paths in cloud, CI/CD and developer tooling | EDR file access on developer hosts and build runners |
| AI-orchestrated collection | The campaign Anthropic reported in November 2025 (ATT&CK C0062), where an AI coding agent automatically collected and processed data, at peak making thousands of requests, often several per second | Database and API logs showing machine-speed request rates |
Platforms are listed alphabetically below where cloud services are named; this is not a ranking.
Turn on the right telemetry first
Automated collection in cloud and SaaS is invisible if object-level and API-level logging is off. Check these before writing analytics.
| Platform | Log source | What it adds for T1119 | Default state and notes |
|---|---|---|---|
| AWS | CloudTrail data events for S3 objects | GetObject, CopyObject and HeadObject calls per principal, the only record of bulk object reads | AWS states CloudTrail does not log data events by default and Event history does not record them; they carry extra charges, so scope them to sensitive buckets if cost is a concern |
| AWS | CloudTrail management events | Configuration harvesting such as describing stacks, instances and IAM | Recorded by default; keep a trail for retention beyond 90 days |
| Microsoft 365 | Purview audit: MailItemsAccessed | Sync (folder download by desktop Outlook) and Bind (individual message access, aggregated per two-minute interval) across POP, IMAP, MAPI, EWS, Exchange ActiveSync and REST | Microsoft states it is enabled by default for users with Office 365 or Microsoft 365 E3/E5 licences |
| Microsoft Entra and Graph | Microsoft Graph activity logs | Every Graph request with AppId, ServicePrincipalId, UserAgent, RequestUri and IPAddress | Requires Entra ID P1 or P2 and diagnostic settings to Log Analytics, Storage or Event Hubs |
| Windows | Sysmon or EDR | Process creation (Event ID 1) and file creation (Event ID 11) used by ATT&CK's Windows analytic | Sysmon needs deployment and a configuration; EDR coverage varies by product |
| Linux | auditd or osquery | execve and open syscalls against document directories by script interpreters | Rules must name the paths and interpreters to watch |
| macOS | Unified log | pbpaste, osascript and Automator activity used for clipboard and file collection | Collect with an EDR or log forwarder |
For storage-specific detection, see the T1530 Data from Cloud Storage Defensive Guide; for mailboxes, the T1114 Email Collection Defensive Guide; for SharePoint, Confluence and similar, the T1213 Data from Information Repositories Defensive Guide.
Detection
ATT&CK's detection strategy for T1119, Automated File and API Collection Detection Across Platforms, has four analytics: Windows process and file bursts, Linux script access to document directories, macOS automation and clipboard tools, and SaaS sign-ins to Graph or sensitive resources from non-browser scripting agents. Each names tunable elements such as the time window, file extension list, process count threshold and expected client IP list. Build from those.
Endpoint signals
- File read bursts: one process opening or copying many files of document types (.docx, .xlsx, .pdf, .pst) across many directories within a short window, especially script hosts and archive tools.
- Staging writes: many file creations in a temporary or unusual directory shortly after the read burst.
- Periodic access: the same process touching the same directory tree at fixed intervals, often launched by a new scheduled task or service.
- Removable media sweeps: a process enumerating newly mounted drives and copying files immediately after insertion.
Cloud and SaaS signals
- Object read rate per principal: GetObject counts per IAM principal and bucket, compared with a 30-day baseline. Sudden reads across many prefixes, or a role that never read objects before, are the strongest signals.
- Non-browser clients on user accounts: Graph or mailbox access from scripting user agents (Python, PowerShell, generic HTTP libraries) for accounts that normally use browsers and Outlook. ATT&CK's SaaS analytic for T1119 uses this, with an expected IP list to suppress known automation.
- Request rate: requests per minute per identity well above human speed, and paging through large result sets (repeated list calls with continuation tokens).
- Mailbox breadth: MailItemsAccessed Bind records covering many folders or a large OperationCount from a new client or IP.
- ETL and pipeline changes: new data pipeline jobs, export tasks or replication rules whose destination is outside your accounts.
Where identity fits
Automated collection usually runs under a non-human identity, or under a human account driven by a script. Both are easier to spot when each identity has a known purpose. Keep an inventory of service principals, access keys and OAuth apps with their expected data scopes, and alert when an identity reads data outside that scope or at a new rate. Over-permissioned read access, such as an app granted tenant-wide mail or file read, turns a single compromised secret into tenant-wide collection. See the Service Account Security Guide and the Non-Human Identity (NHI) Guide.
Prevention and hardening
| Control | Source | How it limits automated collection |
|---|---|---|
| M1029 Remote Data Storage | ATT&CK mitigation | Keeps sensitive data off endpoints where host-based sweeps run |
| M1041 Encrypt Sensitive Information | ATT&CK mitigation | Collected files are useless without keys; ATT&CK notes this may not stop a persistent intruder who reaches the data another way |
| Least privilege for read scopes | NHI Mgmt Group recommendation | Replace tenant-wide read grants with scoped access (specific sites, mailboxes or buckets) for apps and roles |
| Rate limits and quotas | NHI Mgmt Group recommendation | Per-identity API limits on internal data services slow bulk collection and generate signals when hit |
| Data loss prevention | NHI Mgmt Group recommendation | Labels and DLP on sensitive repositories flag bulk access and downloads; see the Data Loss Prevention (DLP) Guide |
| Script control on endpoints | NHI Mgmt Group recommendation | Application control and constrained scripting reduce ad hoc collection scripts on user devices |
Implementing detection in phases
- Enable logging. Turn on S3 data events for sensitive buckets, Graph activity logs and mailbox auditing, and confirm endpoint file telemetry reaches the SIEM.
- Baseline identities. For 30 days, record read volumes and client types per human and non-human identity, and list known automation (backup, eDiscovery, sync, BI tools).
- Deploy in alert-only mode. Start rate and breadth analytics as low-severity findings, tune thresholds and allowlists for known automation, and document each exclusion with an owner.
- Raise severity on combinations. A read burst plus a new scripting client plus a new IP is high severity; any one alone is a hunt lead.
- Automate first response. For high-confidence findings, revoke sessions or disable keys automatically, with a documented rollback (re-enable the key or account after review).
Decision guidance
- Small organisations: prioritise mailbox auditing and SaaS sign-in analytics for non-browser clients, since mail and files are the main targets and logging is often already licensed.
- Large or cloud-heavy organisations: invest in per-principal baselines for storage and Graph, and in scoping data event logging to sensitive data to control cost.
- Precision versus coverage: extension-sweep rules catch commodity stealers cheaply; API-rate baselines catch cloud-native collection but need tuning time.
Response
- Identify the collecting identity. Determine which user, role, key or app made the requests and from where.
- Contain the identity first. Disable the key or app credential, revoke sessions and refresh tokens, and isolate the host if collection was local. See the Workload Identity Compromise Response Playbook for machine identities.
- Scope what was collected. Use object-level logs and MailItemsAccessed records to list the objects and messages accessed; Microsoft notes Sync records mean all items in the folder should be treated as accessed.
- Look for staging and exfiltration. Search for archives, transfers to cloud storage and unusual egress after the collection window.
- Assess notification duties. If personal data was collected, involve privacy and legal teams to assess breach notification obligations.
Detection and measurement
- Logging coverage: share of sensitive buckets with data events enabled and of tenants sending Graph activity logs to the SIEM.
- Baseline coverage: percentage of non-human identities with a documented expected data scope.
- Alert precision: share of T1119 alerts closed as true positive or useful hunt lead, trended monthly.
- Time to contain: median time from a high-severity collection alert to key or session revocation.
- Broad read grants: number of apps or roles with tenant-wide read access to mail, files or storage.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | T1119 Automated Collection; related T1530, T1114, T1213, T1560, T1074, T1059 | The technique and its neighbours in the collection chain |
| ATT&CK mitigations | M1029 Remote Data Storage; M1041 Encrypt Sensitive Information | The two mitigations ATT&CK maps |
| NIST SP 800-53 Rev. 5 | AU-2 Event Logging; AU-6 Audit Record Review; SI-4 System Monitoring; AC-6 Least Privilege; SC-28 Protection of Information at Rest | Logging, monitoring, scoped access and encryption |
| NIST CSF 2.0 | DE.CM-09; DE.AE-02; PR.AA-05; PR.DS-01 | Monitoring, analysis, least privilege and data at rest |
Common failure patterns
- No object-level logging. Without S3 data events or Graph activity logs, bulk reads leave no trace and the scope of a breach cannot be established.
- Allowlisting all automation. Excluding every service principal from analytics to cut noise removes the identities most likely to be used for collection.
- Thresholds without baselines. A fixed "1,000 reads per hour" rule floods on backup jobs and misses a slow collector. Baseline per identity.
- Tenant-wide read grants for apps. One leaked app secret then gives an adversary every mailbox or site.
- Treating developer hosts as low risk. Secret-harvesting malware such as Shai-Hulud sweeps CI/CD and developer tooling paths; build runners and laptops need file access telemetry too.
Practitioner checklist
- Enable S3 data events (or the equivalent object-level logging) for sensitive storage.
- Send Microsoft Graph activity logs and mailbox audit records to the SIEM.
- Deploy endpoint analytics for file read bursts and staging writes by single processes.
- Baseline read volume and client type for every non-human identity with data access.
- Alert on scripting user agents accessing mail and files for interactive user accounts.
- Replace tenant-wide read permissions for apps with scoped grants.
- Document known automation with owners instead of blanket exclusions.
- Write a response runbook that revokes the collecting identity before scoping.
Standards and references
- MITRE ATT&CK: Automated Collection (T1119)
- MITRE ATT&CK: Remote Data Storage (M1029)
- AWS: Amazon S3 CloudTrail Events
- Microsoft Learn: Access Microsoft Graph Activity Logs
- Microsoft Learn: Use MailItemsAccessed to Investigate Compromised Accounts
- Anthropic: Disrupting the First Reported AI-Orchestrated Cyber Espionage Campaign
- NIST: SP 800-53 Rev. 5
Related NHI Mgmt Group resources: ATT&CK Collection Tactic Guide · T1530 Data from Cloud Storage Defensive Guide · Insider Data Theft Guide · UEBA Guide