OpenAI Research: Transfer of Adversarial Robustness Between Perturbation Types
OpenAI researchers have demonstrated that adversarial robustness in deep neural networks does not automatically transfer between different types of perturbations. This finding indicates that a model trained to resist one specific type of attack may remain vulnerable to others, or in some cases, may even become more susceptible to different perturbation types as a result of its training.
Robustness Transfer is Inconsistent Across Perturbation Types
Adversarial robustness does not reliably transfer from one perturbation type to another. While much of the existing research on adversarial examples has focused on $L_{\infty}$ and $L_{2}$-bounded perturbations, these do not encompass the full spectrum of attacks available to an adversary.
To test this, OpenAI evaluated 32 different attacks across five distinct perturbation types against models that were adversarially trained on a 100-class subset of ImageNet. The empirical results indicate that robustness against one specific perturbation type does not guarantee robustness against others.
The Role of Perturbation Size in Evaluation
Understanding whether robustness transfers between perturbation types requires evaluating models across a wide range of perturbation sizes. The researchers found that limiting the evaluation to a narrow set of sizes can lead to an incomplete or misleading understanding of a model's actual resilience.
Potential Trade-offs in Adversarial Defense
Training a model to be robust against one type of perturbation may not only fail to provide protection against other types but can sometimes actively degrade robustness against them. This suggests a potential trade-off where defending against one attack vector may inadvertently create new vulnerabilities to other perturbation types.
Recommendations for AI Safety Evaluation
Because robustness is not universally transferable, OpenAI recommends that the evaluation of adversarial defenses be conducted across a diverse range of perturbation types and sizes. Relying on a single metric or a small set of bounded perturbations (such as $L_{\infty}$ or $L_{2}$) is insufficient to verify the security and reliability of deep neural networks against real-world adversarial threats.