Computer scientists from the National Institute of Standards and Technology (NIST) and their collaborators have published a work titled "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations." This publication aims to shed light on vulnerabilities in artificial intelligence (AI) and machine learning (ML) systems, recognizing the challenges in defending against deliberate manipulations.
AI systems, deeply integrated into modern society, perform various tasks, from autonomous vehicles navigating roads to chatbots engaging with users online. The core issue identified by the researchers is the susceptibility of AI to malicious attacks due to unreliable training data. NIST computer scientist Apostol Vassilev emphasizes the difficulty in ensuring the trustworthiness of data sources, as bad actors can corrupt the data during training and subsequent interactions.
The report categorizes attacks into four major types: evasion, poisoning, privacy, and abuse attacks. Evasion attacks aim to alter inputs after an AI system is deployed, potentially causing it to misinterpret information. Poisoning attacks occur during the training phase, introducing corrupted data to influence the AI's behavior. Privacy attacks focus on extracting sensitive information about the AI or its training data, while abuse attacks involve inserting incorrect information from legitimate sources.
Co-author Alina Oprea, a professor at Northeastern University, highlights the simplicity of executing these attacks with minimal knowledge of the AI system. For instance, poisoning attacks can be mounted by controlling a small percentage of training samples.
The publication provides insights into mitigation strategies for each attack type, emphasizing the lack of foolproof defenses. Vassilev encourages the developer community to contribute to improving defenses against adversarial attacks, emphasizing the need for robust assurances in mitigating risks.
"Despite the significant progress AI and machine learning have made, these technologies are vulnerable to attacks that can cause spectacular failures with dire consequences," warns Vassilev. The report serves as a valuable resource for AI developers and users, fostering awareness of potential risks and the ongoing need for improved defenses in the rapidly evolving field of artificial intelligence.
