Using Non-Linear Activation Functions to increase robustness of AI models to adversarial attacks

Image classification tasks leverage CNN to yield accurate results that supersede their predecessor human-crafted algorithms. Applicable use cases include Autonomous, Face, Medical Imaging, and more. Along with the growing use of AI image classification applications, we see emerging research on the robustness of such models to adversarial attacks, which take advantage of the unique vulnerabilities of the Artificial Intelligence (AI) models to skew their classification results. While not visible to the Human Visual System (HVS), these attacks mislead the algorithms and yield wrong classification results. To be incorporated securely enough in real-world applications, AI-based image classification algorithms require protection that will increase their robustness to adversarial attacks. We propose replacing the commonly used Rectifier Linear Unit (ReLU) Activation Function (AF), which is piecewise linear, with non-linear AF to increase their robustness to adversarial attacks. This approach has been considered in recent research and is motivated by the observation that non-linear AF tends to diminish the effect of adversarial perturbations in the DNN layers. To gain credibility of the approach, we have applied Fast Sign Gradient Method (FGSM), and Hop-Skip-Jump (HSJ) attacks to a trained classification model of the MNIST dataset. We then replaced the AF of the model with non-linear AF (Sigmoid, GeLU, ELU, SeLU, and Tanh). We concluded that while attacks on the original model have a 100% success rate, the attack success rate is dropped by an average of 10% when non-linear AF is used.