Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations

Designing ML models robust in face of supply-chain vulnerabilities is a critical open problem that needs to be addressed by the community.

Fundamentally, the machine learning methodology used in modern AI systems is susceptible to attacks through the public APIs that expose the model, and against the platforms on which they are deployed. This report focuses on the former and considers the latter to be the scope of traditional cybersecurity taxonomies.

SOURCE CODE CONTROL: The attacker might modify the source code of the ML algorithm, such as the random number generator or any third-party libraries, which are often open source.

Image: Adversarial examples of image data modality [120, 288] have the advantage ... of a continuous domain, and gradient-based methods can be applied directly for optimization. Backdoor poisoning attacks were frst invented for images [124], and many privacy attacks are run on image datasets (e.g., [270]). The image modality includes other types of

A taxonomy of the most widely studied and effective attacks in AML, including – evasion, poisoning, and privacy attacks for PredAI systems, – evasion, poisoning, privacy, and abuse/misuse attacks for GenAI systems; ... – attacks against all viable learning methods (e.g., supervised, unsupervised, semisupervised, federated learning, reinforcement le

In a MODEL POISONING attack [185], the adversary controls the model and its parameters.

Adversarial examples became even more intriguing to the research community when Szedegy et al. [288] showed that deep neural networks used for image classifcation can be easily manipulated, and adversarial examples were visualized. In the context of image classifcation, the perturbation of the original sample must be small so that a human cannot ob

In the last few years, many of the proposed mitigations against adversarial examples have been ineffective against stronger attacks. Furthermore, several papers have performed extensive evaluations and defeated a large number of proposed mitigations:

MODEL CONTROL: The attacker might take control of the model parameters by either generating a Trojan trigger and inserting it in the model or by sending malicious local model updates in federated learning.