EvoBA: An Evolution Strategy as a Strong Baseline for Black-Box Adversarial Attacks

Recent work has shown how easily white-box adversarial attacks can be applied to state-of-the-art image classifiers. However, real-life scenarios resemble more the black-box adversarial conditions, lacking transparency and usually imposing natural, hard constraints on the query budget. It is usual for the black-box adversarial literature to assume that the attacker, while constrained by the number of model invocations and total image perturbation, has huge resources in terms of computation power and time. Therefore, most black-box adversarial attacks take a long time to run and require heavy computational processes. This makes it unfeasible for day-to-day computer vision systems developers to run continuous robustness checks against their models, by playing the attacker role and seeing when and how their model fails. Therefore, we propose a class of black-box attacks based on simple evolutionary strategies: $\textbf{EvoBA}$, $\textbf{EvoBA-E}$, and $\textbf{EvoBA-ET}$. These are fast, query-efficient attacks, aiming to minimize the $L_0$ adversarial perturbations, and do not require any form of training. We benchmark these attacks on the CIFAR-100 dataset and observe that they bear different trade-offs in terms of the count of queries required to fool the models and the $L_0$ adversarial distance. However, $\textbf{EvoBA}$ strikes out as being the fastest of the three, so we further focus on comparing it against well-known adversarial attacks. We also build and open-source \footnote{https://github.com/andreiilie1/BBAttacks} a tool that can be used to run quick robustness checks of generic computer vision classifier systems. $\textbf{EvoBA}$ shows efficiency and efficacy through results that are in line with much more complex state-of-the-art black-box attacks such as $\textbf{AutoZOOM}$. It is more query-efficient than $\textbf{SimBA}$, a simple and powerful baseline black-box attack, and has a similar level of complexity. Therefore, we propose it both as a new strong baseline for black-box adversarial attacks and as a fast and general tool for gaining empirical insight into how robust image classifiers are with respect to $L_0$ adversarial perturbations. There exist fast and reliable $L_2$ black-box attacks, such as $\textbf{SimBA}$, and $L_{\infty}$ black-box attacks, such as $\textbf{DeepSearch}$. We propose $\textbf{EvoBA}$ as a query-efficient $L_0$ black-box adversarial attack which, together with the aforementioned methods, can serve as a generic tool to assess the empirical robustness of image classifiers. The main advantages of such methods are that they run fast, are query-efficient, and can easily be integrated in image classifiers development pipelines. While our attack minimises the $L_0$ adversarial perturbation, we also report $L_2$, and notice that we compare favorably to the state-of-the-art $L_2$ black-box attack, $\textbf{AutoZOOM}$, and of the $L_2$ strong baseline, $\textbf{SimBA}$.