Skip to content

MITRE ATLAS Course of Action

MITRE ATLAS Mitigation - Adversarial Threat Landscape for Artificial-Intelligence Systems

Authors
Authors and/or Contributors
MITRE

Limit Release of Public Information

Limit the public release of technical information about the machine learning stack used in an organization's products or services. Technical knowledge of how machine learning is used can be leveraged by adversaries to perform targeting and tailor attacks to the target system. Additionally, consider limiting the release of organizational information - including physical locations, researcher names, and department structures - from which technical details such as machine learning techniques, model architectures, or datasets may be inferred.

Internal MISP references

UUID 0b016f6f-2f61-493c-bf9d-02cad4c027df which can be used as unique global reference for Limit Release of Public Information in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0000
Related clusters

To see the related clusters, click here.

Limit Model Artifact Release

Limit public release of technical project details including data, algorithms, model architectures, and model checkpoints that are used in production, or that are representative of those used in production.

Internal MISP references

UUID c0f65fa8-8e05-4481-b934-ff2c452ae8c3 which can be used as unique global reference for Limit Model Artifact Release in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0001
Related clusters

To see the related clusters, click here.

Passive ML Output Obfuscation

Decreasing the fidelity of model outputs provided to the end user can reduce an adversaries ability to extract information about the model and optimize attacks for the model.

Internal MISP references

UUID 6b53cb14-eade-4760-8dae-75164e62cb7e which can be used as unique global reference for Passive ML Output Obfuscation in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0002
Related clusters

To see the related clusters, click here.

Model Hardening

Use techniques to make machine learning models robust to adversarial inputs such as adversarial training or network distillation.

Internal MISP references

UUID 04e9bb75-1b7e-4825-bc3f-774850d3c1ef which can be used as unique global reference for Model Hardening in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0003
Related clusters

To see the related clusters, click here.

Restrict Number of ML Model Queries

Limit the total number and rate of queries a user can perform.

Internal MISP references

UUID 4a048bfe-dab5-434b-86cc-f4586951ec0d which can be used as unique global reference for Restrict Number of ML Model Queries in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0004
Related clusters

To see the related clusters, click here.

Control Access to ML Models and Data at Rest

Establish access controls on internal model registries and limit internal access to production models. Limit access to training data only to approved users.

Internal MISP references

UUID da785068-ece5-4c52-b77d-39e1b24cb6d7 which can be used as unique global reference for Control Access to ML Models and Data at Rest in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0005
Related clusters

To see the related clusters, click here.

Use Ensemble Methods

Use an ensemble of models for inference to increase robustness to adversarial inputs. Some attacks may effectively evade one model or model family but be ineffective against others.

Internal MISP references

UUID de7a696b-f688-454c-bf61-476a68b50e9f which can be used as unique global reference for Use Ensemble Methods in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0006
Related clusters

To see the related clusters, click here.

Sanitize Training Data

Detect and remove or remediate poisoned training data. Training data should be sanitized prior to model training and recurrently for an active learning model.

Implement a filter to limit ingested training data. Establish a content policy that would remove unwanted content such as certain explicit or offensive language from being used.

Internal MISP references

UUID 7e20b527-6299-4ee3-863e-59fee7cdaa9a which can be used as unique global reference for Sanitize Training Data in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0007
Related clusters

To see the related clusters, click here.

Validate ML Model

Validate that machine learning models perform as intended by testing for backdoor triggers or adversarial bias.

Internal MISP references

UUID 32bd077a-90ce-4e97-ad40-8f130a1a7dab which can be used as unique global reference for Validate ML Model in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0008
Related clusters

To see the related clusters, click here.

Use Multi-Modal Sensors

Incorporate multiple sensors to integrate varying perspectives and modalities to avoid a single point of failure susceptible to physical attacks.

Internal MISP references

UUID 532918ce-83cf-4f6f-86fa-8ad4024e91ab which can be used as unique global reference for Use Multi-Modal Sensors in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0009
Related clusters

To see the related clusters, click here.

Input Restoration

Preprocess all inference data to nullify or reverse potential adversarial perturbations.

Internal MISP references

UUID 88aea80f-498f-403d-b82f-e76c44f9da94 which can be used as unique global reference for Input Restoration in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0010
Related clusters

To see the related clusters, click here.

Restrict Library Loading

Prevent abuse of library loading mechanisms in the operating system and software to load untrusted code by configuring appropriate library loading mechanisms and investigating potential vulnerable software.

File formats such as pickle files that are commonly used to store machine learning models can contain exploits that allow for loading of malicious libraries.

Internal MISP references

UUID 6cd8c9ca-bd46-489f-9ccb-5b76b8ef580e which can be used as unique global reference for Restrict Library Loading in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0011
Related clusters

To see the related clusters, click here.

Encrypt Sensitive Information

Encrypt sensitive data such as ML models to protect against adversaries attempting to access sensitive data.

Internal MISP references

UUID 8bba19a7-fc6f-4381-8b34-2d43cdc14627 which can be used as unique global reference for Encrypt Sensitive Information in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0012
Related clusters

To see the related clusters, click here.

Code Signing

Enforce binary and application integrity with digital signature verification to prevent untrusted code from executing. Adversaries can embed malicious code in ML software or models. Enforcement of code signing can prevent the compromise of the machine learning supply chain and prevent execution of malicious code.

Internal MISP references

UUID c55ed072-eca7-41d6-b5e0-68c10753544d which can be used as unique global reference for Code Signing in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0013
Related clusters

To see the related clusters, click here.

Verify ML Artifacts

Verify the cryptographic checksum of all machine learning artifacts to verify that the file was not modified by an attacker.

Internal MISP references

UUID a861f658-4203-48ba-bdca-fe068518eefb which can be used as unique global reference for Verify ML Artifacts in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0014
Related clusters

To see the related clusters, click here.

Adversarial Input Detection

Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the ML system prior to the ML model.

Internal MISP references

UUID 825f21ab-f3c9-46ce-b539-28f295f519f8 which can be used as unique global reference for Adversarial Input Detection in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0015
Related clusters

To see the related clusters, click here.

Vulnerability Scanning

Vulnerability scanning is used to find potentially exploitable software vulnerabilities to remediate them.

File formats such as pickle files that are commonly used to store machine learning models can contain exploits that allow for arbitrary code execution.

Internal MISP references

UUID e2cb599d-2714-4673-bc1a-976c471d7c58 which can be used as unique global reference for Vulnerability Scanning in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0016
Related clusters

To see the related clusters, click here.

Model Distribution Methods

Deploying ML models to edge devices can increase the attack surface of the system. Consider serving models in the cloud to reduce the level of access the adversary has to the model.

Internal MISP references

UUID 79316871-3bf9-4a59-b517-b0156e84fcb4 which can be used as unique global reference for Model Distribution Methods in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0017
Related clusters

To see the related clusters, click here.

User Training

Educate ML model developers on secure coding practices and ML vulnerabilities.

Internal MISP references

UUID 8c2cb25a-46b0-4551-beeb-21e8425a48bd which can be used as unique global reference for User Training in MISP communities and other software using the MISP galaxy

External references
Associated metadata
Metadata key Value
external_id AML.M0018
Related clusters

To see the related clusters, click here.