Poisoning Attacks (5d6df7ca-c0e7-5530-9dce-22a92e7c103a)
In a poisoning attack, the goal of the attacker is to contaminate the training data or the model generated in the training phase, so that predictions on new data will be modified in the testing phase. This attack could also be caused by insiders. Example: in a medical dataset where the goal is to predict the dosage of a medicine using demographic information, researchers introduced malicious samples at 8% poisoning rate, which changed the dosage by 75.06% for half of the patients.
Other scenarios: * Data tampering: Actors like AI/ML designers and engineers can deliberately or unintentionally manipulate and expose data. Data can also be manipulated during the storage procedure and by means of some processes like feature selection. Besides interfering with model inference, this type of threat can also bring severe discriminatory issues by introducing bias. Source: ENISA * An attacker who knows how a raw data filtration scheme is set up may be able to leverage that knowledge into malicious input later in system deployment. Source:BerryVilleiML * Adversaries may fine-tune hyper-parameters and thus influence the AI system’s behavior. Hyper-parameters can be a vector for accidental overfitting. In addition, hard to detect changes to hyper-parameters would make an ideal insider attack. Source: ENISA
Threat-modeling question: Are we protected from poisoning attacks?