MITRE ATLAS Course of Action
MITRE ATLAS Mitigation - Adversarial Threat Landscape for Artificial-Intelligence Systems
Authors
| Authors and/or Contributors |
|---|
| MITRE |
Limit Release of Public Information
Limit the public release of technical information about the machine learning stack used in an organization's products or services. Technical knowledge of how machine learning is used can be leveraged by adversaries to perform targeting and tailor attacks to the target system. Additionally, consider limiting the release of organizational information - including physical locations, researcher names, and department structures - from which technical details such as machine learning techniques, model architectures, or datasets may be inferred.
Internal MISP references
UUID 0b016f6f-2f61-493c-bf9d-02cad4c027df which can be used as unique global reference for Limit Release of Public Information in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0000 |
Related clusters
To see the related clusters, click here.
Limit Model Artifact Release
Limit public release of technical project details including data, algorithms, model architectures, and model checkpoints that are used in production, or that are representative of those used in production.
Internal MISP references
UUID c0f65fa8-8e05-4481-b934-ff2c452ae8c3 which can be used as unique global reference for Limit Model Artifact Release in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0001 |
Related clusters
To see the related clusters, click here.
Passive ML Output Obfuscation
Decreasing the fidelity of model outputs provided to the end user can reduce an adversaries ability to extract information about the model and optimize attacks for the model.
Internal MISP references
UUID 6b53cb14-eade-4760-8dae-75164e62cb7e which can be used as unique global reference for Passive ML Output Obfuscation in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0002 |
Related clusters
To see the related clusters, click here.
Model Hardening
Use techniques to make machine learning models robust to adversarial inputs such as adversarial training or network distillation.
Internal MISP references
UUID 04e9bb75-1b7e-4825-bc3f-774850d3c1ef which can be used as unique global reference for Model Hardening in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0003 |
Related clusters
To see the related clusters, click here.
Restrict Number of ML Model Queries
Limit the total number and rate of queries a user can perform.
Internal MISP references
UUID 4a048bfe-dab5-434b-86cc-f4586951ec0d which can be used as unique global reference for Restrict Number of ML Model Queries in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0004 |
Related clusters
To see the related clusters, click here.
Control Access to ML Models and Data at Rest
Establish access controls on internal model registries and limit internal access to production models. Limit access to training data only to approved users.
Internal MISP references
UUID da785068-ece5-4c52-b77d-39e1b24cb6d7 which can be used as unique global reference for Control Access to ML Models and Data at Rest in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0005 |
Related clusters
To see the related clusters, click here.
Use Ensemble Methods
Use an ensemble of models for inference to increase robustness to adversarial inputs. Some attacks may effectively evade one model or model family but be ineffective against others.
Internal MISP references
UUID de7a696b-f688-454c-bf61-476a68b50e9f which can be used as unique global reference for Use Ensemble Methods in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0006 |
Related clusters
To see the related clusters, click here.
Sanitize Training Data
Detect and remove or remediate poisoned training data. Training data should be sanitized prior to model training and recurrently for an active learning model.
Implement a filter to limit ingested training data. Establish a content policy that would remove unwanted content such as certain explicit or offensive language from being used.
Internal MISP references
UUID 7e20b527-6299-4ee3-863e-59fee7cdaa9a which can be used as unique global reference for Sanitize Training Data in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0007 |
Related clusters
To see the related clusters, click here.
Validate ML Model
Validate that machine learning models perform as intended by testing for backdoor triggers or adversarial bias.
Internal MISP references
UUID 32bd077a-90ce-4e97-ad40-8f130a1a7dab which can be used as unique global reference for Validate ML Model in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0008 |
Related clusters
To see the related clusters, click here.
Use Multi-Modal Sensors
Incorporate multiple sensors to integrate varying perspectives and modalities to avoid a single point of failure susceptible to physical attacks.
Internal MISP references
UUID 532918ce-83cf-4f6f-86fa-8ad4024e91ab which can be used as unique global reference for Use Multi-Modal Sensors in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0009 |
Related clusters
To see the related clusters, click here.
Input Restoration
Preprocess all inference data to nullify or reverse potential adversarial perturbations.
Internal MISP references
UUID 88aea80f-498f-403d-b82f-e76c44f9da94 which can be used as unique global reference for Input Restoration in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0010 |
Related clusters
To see the related clusters, click here.
Restrict Library Loading
Prevent abuse of library loading mechanisms in the operating system and software to load untrusted code by configuring appropriate library loading mechanisms and investigating potential vulnerable software.
File formats such as pickle files that are commonly used to store machine learning models can contain exploits that allow for loading of malicious libraries.
Internal MISP references
UUID 6cd8c9ca-bd46-489f-9ccb-5b76b8ef580e which can be used as unique global reference for Restrict Library Loading in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0011 |
Related clusters
To see the related clusters, click here.
Encrypt Sensitive Information
Encrypt sensitive data such as ML models to protect against adversaries attempting to access sensitive data.
Internal MISP references
UUID 8bba19a7-fc6f-4381-8b34-2d43cdc14627 which can be used as unique global reference for Encrypt Sensitive Information in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0012 |
Related clusters
To see the related clusters, click here.
Code Signing
Enforce binary and application integrity with digital signature verification to prevent untrusted code from executing. Adversaries can embed malicious code in ML software or models. Enforcement of code signing can prevent the compromise of the machine learning supply chain and prevent execution of malicious code.
Internal MISP references
UUID c55ed072-eca7-41d6-b5e0-68c10753544d which can be used as unique global reference for Code Signing in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0013 |
Related clusters
To see the related clusters, click here.
Verify ML Artifacts
Verify the cryptographic checksum of all machine learning artifacts to verify that the file was not modified by an attacker.
Internal MISP references
UUID a861f658-4203-48ba-bdca-fe068518eefb which can be used as unique global reference for Verify ML Artifacts in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0014 |
Related clusters
To see the related clusters, click here.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the ML system prior to the ML model.
Internal MISP references
UUID 825f21ab-f3c9-46ce-b539-28f295f519f8 which can be used as unique global reference for Adversarial Input Detection in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0015 |
Related clusters
To see the related clusters, click here.
Vulnerability Scanning
Vulnerability scanning is used to find potentially exploitable software vulnerabilities to remediate them.
File formats such as pickle files that are commonly used to store machine learning models can contain exploits that allow for arbitrary code execution.
Internal MISP references
UUID e2cb599d-2714-4673-bc1a-976c471d7c58 which can be used as unique global reference for Vulnerability Scanning in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0016 |
Related clusters
To see the related clusters, click here.
Model Distribution Methods
Deploying ML models to edge devices can increase the attack surface of the system. Consider serving models in the cloud to reduce the level of access the adversary has to the model.
Internal MISP references
UUID 79316871-3bf9-4a59-b517-b0156e84fcb4 which can be used as unique global reference for Model Distribution Methods in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0017 |
Related clusters
To see the related clusters, click here.
User Training
Educate ML model developers on secure coding practices and ML vulnerabilities.
Internal MISP references
UUID 8c2cb25a-46b0-4551-beeb-21e8425a48bd which can be used as unique global reference for User Training in MISP communities and other software using the MISP galaxy
External references
Associated metadata
| Metadata key | Value |
|---|---|
| external_id | AML.M0018 |
Related clusters
To see the related clusters, click here.