Every app you publish to a store, every public code repository and every reachable container registry tells an adversary whether you use AI, which framework you use and, often, where the model file sits. MITRE ATLAS records this as AML.T0004 Search Application Repositories, a Reconnaissance (AML.TA0002) technique rated Demonstrated. ATLAS names Google Play, the iOS App Store, the macOS App Store and the Microsoft Store as examples, and its case studies also map the technique to searches of source code repositories and exposed container registries. This guide covers the AI-specific angle: what an adversary learns about your models from published packages, the cases ATLAS records, what you can and cannot detect, and how to reduce the exposure. It is for mobile and ML engineers, AI platform owners and security teams that ship AI features in apps or images. The wider reconnaissance tactic is covered in MITRE ATLAS AI Reconnaissance and Resource Development; this page does not repeat it.
Key takeaways
- The search is invisible to you. Browsing a public app store or code host generates no log you own, so defence has to focus on what your published artefacts contain.
- The usual next step is taking the model. ATLAS says AML.T0004 is frequently followed by AML.T0002 Acquire Public AI Artifacts, and its mobile app case went from a keyword search to full model access.
- On-device models are often unprotected. A study of 1,468 machine learning apps found 41% did not protect their models at all.
- ATLAS maps one mitigation. AML.M0000 Limit Public Release of Information: keep sensitive detail out of the metadata of deployed systems and public applications.
- Do the adversary's search first. Regular self-reconnaissance of your own store listings, packages, repositories and registries is the most practical control and the easiest to measure.
What the technique is
ATLAS describes AML.T0004 as searching open application repositories during targeting, with queries crafted to find applications that contain AI-enabled components. The technique entry lists the Enterprise platform; it was created in May 2021 and last modified in May 2026. ATLAS adds that the next step is frequently Acquire Public AI Artifacts (AML.T0002), in particular its Models sub-technique (AML.T0002.001).
The ATLAS case studies (2026.10 release) apply the technique to three kinds of repository. They are worth separating because each leaks different information and needs a different owner to fix it.
| Repository | What signals AI use | What the adversary can learn | Who owns the fix |
|---|---|---|---|
| App stores (Google Play, Apple App Store, Microsoft Store) | Listing text that advertises AI features; inference framework libraries and model files inside the package | The framework, the model file and format, embedded label files, and any API keys used to reach AI services | Mobile and desktop app teams, release engineering |
| Public source code repositories | Dependencies on LLM frameworks, deployment URLs, configuration files | Which framework functions are called, where the app is deployed, and how prompts and tools are wired together | Development teams, open source programme office |
| Container registries exposed to the internet | Images containing model files and inference servers | Complete models, and on misconfigured registries the ability to replace images | Platform and DevOps teams |
Searching victim-owned websites (AML.T0003) and open websites and domains (AML.T0095) are separate ATLAS techniques; see AML.T0003 Search Victim-Owned Websites and AML.T0095 Search Open Websites/Domains. Within ATT&CK, the closest Enterprise analogue for the code repository part is T1593.003 Code Repositories, under T1593 Search Open Websites/Domains; ATT&CK has no technique for app store searches.
Cases in ATLAS and published research
Backdoored models in mobile apps (AML.CS0013)
Researchers searched Google Play for apps likely to contain embedded deep learning models using deep learning keywords (AML.T0004). They then filtered the packages by searching code metadata for TensorFlow and TFLite keywords and model file formats, and extracted the models from the APKs (AML.T0002.001). That gave full access to the models in compiled form (AML.T0044). The rest of the case shows why reconnaissance matters: the researchers injected a neural payload into the compiled model, repackaged the app and showed that a physical visual trigger could bypass the model in safety-critical apps. The underlying paper, DeepPayload, reports 54 vulnerable apps found and a 93.5% trigger success rate, with latency overhead under 2 ms and an accuracy drop of no more than 1.4%.
Exposed container registries holding models (AML.CS0028)
Trend Micro researchers used service indexing portals and web search tools to find private container registries exposed on the internet, which ATLAS maps to AML.T0004. Their report counts 8,817 registries reachable without authentication, of which 6,259 (about 70%) allowed pushes, and 1,453 unique AI models inside the images, almost half in ONNX format. ATLAS records the follow-on risk: because many registries allowed writes, an adversary could push a manipulated model under the same name and tag, and automated pipelines could pull it (AML.T0010.004 Container Registry).
RCE in LLM-integrated apps (AML.CS0052, LLMSmith)
Researchers first analysed 11 LLM frameworks for functions that execute code from user input or model output, then, per ATLAS, scanned source code repositories for app deployment URLs to find applications likely built with those frameworks (AML.T0004). The paper reports 20 vulnerabilities (19 RCE, one arbitrary file read or write), 11 CVE IDs, and successful attacks on 17 of 51 potentially affected apps. The later steps run through AML.T0049 Exploit Public-Facing Application and AML.T0053 AI Agent Tool Invocation.
How common unprotected on-device models are
The study "Mind Your Weight(s)" analysed 46,753 popular apps from US and Chinese app markets, of which 1,468 used machine learning. It found that 41% of the ML apps did not protect their models at all, and that models could be extracted from 66% of the apps that did use protection or encryption, with unsophisticated dynamic analysis. The extracted models were mostly commercial models for face recognition, liveness detection, ID and bank card recognition and malware detection.
Where identity and secrets fit
An app package that calls a hosted AI service has to authenticate somehow. If it carries a static API key, the same search that finds the AI feature finds the key, and the key is a non-human identity with whatever scope it was given. Symantec's 2022 research shows the general pattern for cloud keys: the Mobile Apps Hard-Coded AWS Keys 2022 page covers 1,859 apps carrying AWS credentials, many shared across developers through common SDKs. Treat any key shipped in a public package as public. Route AI service calls through your own backend, issue short-lived, narrowly scoped tokens to the app, and enforce per-identity quotas at the gateway. See How to Protect API Keys in Mobile Apps and Secrets in mobile apps. Container registries are the same problem at the infrastructure layer: registry credentials and pull tokens are machine identities, and an anonymous push permission is the most dangerous one of all.
Reducing what your repositories reveal
ATLAS maps one mitigation to this technique: AML.M0000 Limit Public Release of Information. For AML.T0004 it says to limit the release of sensitive information in the metadata of deployed systems and publicly available applications. The table turns that into specific controls.
| Control | Where | Detail |
|---|---|---|
| Keep high-value models server side | Architecture | If a model is a trade secret or a security control (fraud, liveness, malware detection), serve it through an authenticated API rather than shipping weights to every device |
| Protect models that must ship | Mobile and desktop builds | Encrypt model files, decrypt only in memory, and add integrity checks. OWASP MASVS-RESILIENCE lists theft of AI or machine learning models among the risks resilience controls address, while warning they do not replace sound security architecture |
| Strip identifying metadata | Build pipeline | Remove descriptive model file names, training notes, label files the client does not need and debug symbols that name frameworks and model versions (AML.M0000) |
| No static AI service keys in packages | App and backend | Backend broker with short-lived tokens; per-identity rate limits and quotas |
| Review public repositories before release | Source control | Secret scanning plus a check for deployment URLs, internal endpoints, prompts and agent tool configuration that should not be public |
| Authenticate every registry | Container platform | Authentication on from the start, including test registries; restrict network reachability; audit logging; no anonymous push |
| Verify model integrity at deploy | CI/CD and runtime | Sign models, compare checksums against the released version, and keep models versioned so a tampered one can be rolled back |
The registry and integrity controls follow the recommendations Trend Micro made alongside its registry research. Model signing and inventory are covered in the AI Supply Chain and AI-BOM Guide.
Decision guidance: on-device inference is often the right choice for privacy, latency or offline use, and hiding the fact that an app uses AI is rarely realistic or necessary. The question is what an extracted model is worth to an adversary. A photo filter model is low value; a liveness or fraud model that gates account access is a security control, and an adversary with a full copy can craft inputs against it offline. Small teams should start with the two cheapest wins: no static keys in packages, and no unauthenticated registries. Larger organisations with many apps should add an inventory of which published packages contain which models.
What to detect and where
The reconnaissance itself happens on third-party platforms and leaves nothing in your logs. Detection therefore watches the steps that depend on it and your own exposure.
| Where | What to watch | Why it relates to AML.T0004 |
|---|---|---|
| Your published packages | Scheduled scans of release builds for model files, label maps, framework libraries and credentials | Shows exactly what a searcher would find |
| App stores | Repackaged or cloned copies of your app under other publishers | CS0013 relied on repackaging an app with a modified model |
| AI service gateway | Calls from a key issued to an app but arriving from server infrastructure, unusual volume or new regions | Signals a key extracted from a package |
| Container registries | Anonymous pulls, pushes from unknown identities, tags overwritten on model images | The CS0028 follow-on step |
| Public code hosts | Secret scanning alerts, new public repositories under your organisation | Deployment URLs and configuration feed CS0052 style targeting |
| Model serving | Hash of the loaded model against the released version | Detects a replaced model after an extraction and repackage |
How to test
- Inventory: list every app, SDK, public repository and container registry your organisation publishes, and which of them include AI features.
- Self-reconnaissance: search the stores and code hosts the way an outsider would, using your brand plus AI and framework terms, and record what turns up.
- Package review: unpack your own release builds in a controlled environment and record model files, formats, label files, framework strings and any credentials. The Mobile App Security Testing Guide covers safe test setup.
- Registry check: confirm from outside your network that no registry accepts anonymous pulls or pushes.
- Fix and re-test: apply the controls, rebuild, repeat the review and confirm the findings are gone. Revoke and rotate any credential found, rather than only removing it from the next build.
Measuring it
- Share of published app packages that contain an unencrypted model file (target: only models classed as low value).
- Number of static AI service or cloud credentials found in release builds per quarter (target zero).
- Number of internet-reachable registries accepting anonymous access (target zero).
- Share of deployed model images verified by signature or checksum at deploy time.
- Time from a secret scanning alert on a public repository to credential revocation.
Common failure patterns
- Shipping the crown jewels: a liveness or fraud model shipped in clear in the app, so a single download hands it over.
- Encryption with the key beside it: models encrypted at rest but decrypted with a key held in the same package, which the "Mind Your Weight(s)" study shows is easy to defeat.
- Keys in the build: AI service keys compiled into the app for convenience and never rotated, as in the hard-coded cloud keys Symantec found.
- Test registries left open: registries set up without authentication for testing that stay reachable, including write access.
- Revoke-free clean-up: removing a leaked key from the next release while the old one stays valid in every copy already downloaded.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| MITRE ATLAS | AML.T0004 (AML.TA0002 Reconnaissance) | The technique; often followed by AML.T0002.001 Models and AML.T0044 Full AI Model Access |
| MITRE ATLAS mitigation | AML.M0000 Limit Public Release of Information | The only mitigation ATLAS maps to this technique |
| MITRE ATLAS case studies | AML.CS0013, AML.CS0028, AML.CS0052 | App store, container registry and code repository searches |
| MITRE ATT&CK | T1593.003 Code Repositories | Closest analogue for the source code part |
| OWASP MASVS | MASVS-RESILIENCE | Reverse engineering and tampering resilience, including AI model theft |
| OWASP Top 10 for LLM Applications 2025 | LLM03 Supply Chain | Tampered models and third-party components entering the pipeline |
| NIST SP 800-53 Rev. 5 | IA-5, SC-28, AC-3 | Authenticator management for embedded keys, protection of model files at rest, access enforcement on registries |
Practitioner checklist
- Inventory every published app, SDK, repository and registry that contains AI components.
- Run self-reconnaissance on store listings and public code at least each release cycle.
- Keep high-value models server side; encrypt and integrity-check models that must ship.
- Strip model names, label files and framework detail the client does not need.
- Remove static AI service keys from packages; broker calls through a backend with short-lived tokens.
- Require authentication on every container registry and block anonymous push.
- Sign models and verify hashes at deploy so a replaced model is caught.
- Revoke and rotate any credential found in a published artefact immediately.
Standards and references
- MITRE ATLAS: Search Application Repositories
- MITRE ATLAS: Backdoor Attack on Deep Learning Models in Mobile Apps
- MITRE ATLAS: LLMSmith: RCE Vulnerabilities in LLM-Integrated Applications
- OWASP: MASVS-RESILIENCE
- Sun et al.: Mind Your Weight(s): A Large-scale Study on Insufficient Machine Learning Model Protection in Mobile Apps
- Li et al.: DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection
- Trend Micro: Silent Sabotage: Weaponizing AI Models in Exposed Containers
- Liu et al.: Demystifying RCE Vulnerabilities in LLM-Integrated Apps
Related NHI Mgmt Group resources: MITRE ATLAS Guide · AML.T0007 Discover AI Artifacts · Mobile App Security Guide · Model Extraction and Distillation Defence Guide