Adversarial Attacks & Defenses at CVPR 2019
Analyzing the trends in Adversarial A&D at the IEEE Conference on Computer Vision and Pattern Recognition 2019. This post is supported by a more in-depth summary of papers published at CVPR over the years here.
In this post, I will be presenting the recent trends in adversarial Attacks and Defenses at CVPR’19. I hope this high-level analysis increases your awareness of the exciting new results, and perhaps motivate you to pursue specific directions within this field. TL;DR key-points are: (a) better attacks and significantly stronger defense strategies than CVPR’18, (b) new tasks have been attempted such as targeted caption generation, and ( c ) Scores of papers on attacking 3D data such as meshes and point clouds.
A brief look at the news would inform you about the boom in the field of Artificial Intelligence, be it in research or application. With such a exponential growth in this field, problems which were (re)introduced only a few years ago have became vast sub-fields in themselves . This post deals with one such topic: Adversarial Attacks and Defenses. While this sub-field expands beyond the domain of vision; in this post, we will be looking at the recent trends of Adversarial Attacks & Defenses by exploring one of the most impact computer vision conferences: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Note: This post assumes a basic understanding of adversarial attacks & defenses. Check out these amazing blogs (a, b, and c) if you are unfamiliar with this topic!
Increasing papers on Adversarial Attacks & Defenses
The graph above shows the overall number of papers and the number of papers related to adversarial A&D accepted at CVPR over the past 4 years. For adversarial A&D, from 8 papers at CVPR’18 to 33 papers at CVPR’19, there is an impressive growth. While the total paper count has increased by ~30%, papers in adversarial A&D increased by over 400%! Furthermore, 4 different workshops related to adversarial A&D are being held at CVPR’19. What this trend indicates is the increase in the research input, from academia as well as industry. Having invested a sizable amount of time and effort during the past 2 years in pursuing research ideas in this field, I have frequently considered the following question: Why is the academia, private industry and government investing time, efforts and money in this field?
Get Aditya Ganeshan’s stories in your inbox
Join Medium for free to get updates from this writer.
I consider it to be broadly due to two dominant factors:
- Exposing and assuaging the threat to AI Solutions deployed in critical procedures: For deploying a deep-learning based solution in critical procedures such as autonomous driving or surgical systems, it is essential to address its susceptibility to adversarial perturbations. Systems participating in such critical procedures need strong security measures against disruptive acts from nefarious agents.
- Exposing the gap in our current understanding of ‘learning’: Current neural networks architectures are only weakly modeled after our biological neural systems. However, our current learning algorithms are our best approximations to how humans ‘learn’. Adversarial attacks expose the inherent difference between model fitting and the human ‘learning’ as we ourselves do not face this problem (although we do have a different set of adversaries: visual illusions).
While curiosity definitely plays a role, it’s undeniable that the potential impact of this field acts as a motivating factor as well. Now, its time to look into what these research inputs have led to.
Trends in CVPR 2019
The above pie chart distributes all the 33 papers into 5 semantically similar bins. A big chunk of works are centered around defense against adversarial attacks. Surprisingly, there are 6 new works on attacking 3D data such as meshes, voxels etc!
At CVPR’19, more than 7 new attacks have been proposed which deal with image classification. Works such as [22, 24, 27] introduced improved optimization for crafting perturbations. SparseFool, a method from Dezfooli et al., builds on top of their previous attack methods DeepFool, producing sparse yet lethal perturbations. In Curl and Whey, Shi et al. introduce a novel idea that ‘curling’, i.e. using gradient descent as well as ascent, can yield stronger black-box perturbations (quick timeout: White-box attack refers to attacks with intrinsic knowledge of the system; in this case, parameters of the neural network and black-box attacks refer to attacks without such knowledge). When combined with a denoising ‘whey’ iteration, this method yields one of the strongest black box attacks known. Curl and Whey perturbations achieve comparable results to existing attacks while being ~30% smaller than other perturbations. Finally, in Catastrophic Child’s Play, Ho et al. propose an attack formulation where the object or the camera is manipulated in real life, and show that none of the existing defense mechanisms sufficiently improve predictions from such images.
When it comes to defenses, the works are aplenty. Significant strides have been made in defense against white as well as black-box attack in works such as Feature Denoising from Xie et al. and Feature Distillation from Liu et al. For example, Feature Denoising increases top-1 accuracy of Resnet-152 against strong white-box attacks from 27.9% to 55.7%! New methods such as Stratified Convolutional Sparse Coding, and ComDefend seek to denoise/reproject the adversarial image to clean image manifold before passing it to the target network. In contrast, works such as Parametric Noise Injection present novel methods for adversarial training. Finally, few of the papers such as BaRT from Raff et al. and Randomized Diversification from Taran et al. utilize randomization as a key component of their defense strategy.
A big change this year has been regarding the different types of tasks that have been attacked. In [33], authors Xu et al. present novel optimization methods for yielding target partial captions (as shown in the adjacent image) from images. Works such as Fooling automated surveillance cameras from Thys et al. yield a printable patch which enables the person holding the patch to avoid detection from any angle and position (results in the video below). Further, [30, 34] leverage Image Retrieval to provide an effective defense against black box attacks. Finally, in [31], Dong et al. present a powerful evolutionary algorithm for attacking Face Recognition systems in a black box setting. This is further utilized to successfully attack TenCent’s Face Verification Platform.
While the above works primarily present an attack or a defense, works such as [41, 42, 43] explore the underlying reasons for the existence of adversarial A&D. In [41], Corneanu et al. provide a novel definition of what ‘learning’ actually means. In contrast, works such as [42, 43], focus on the learnt space and decision boundaries.
Finally, there has been a deluge of interest in attacking 3D data. This has definitely been enabled by the invention of differentiable rendering techniques in the past few years. While about half the works [36, 37, 38, 39] attack the system by changing the object structure itself (mesh or voxel or point cloud), others [35, 40] craft adversaries by changing illumination or pose of object. These works open up multiple methods to attack 3D data and raise concerns regarding utilizing 3D data reliant pipelines in critical activities such as autonomous driving.
Some of the other minor trends are shown below in the chart:
Closing Remark:
In day to day research, we are generally bogged down by specific problems in specific domains. I hope this mini-survey helps you to take a broader view of adversarial A&D and the recent exciting works. Maybe you will join some dots between your research and one of the works mentioned above, and take a leap into adversarial A&D as well! I will be very interested to know which paper you liked the most and why.
I am keeping a more in-depth summary of each of the adversarial A&D work at CVPR at this repository. Check it out for more detailed, yet less wordy paper summaries (i.e. ~5–7 lines per paper). I am still only halfway through with the papers, but hopefully it will be finished soon.
I’d be thankful if you inform me about any errors in this post. I’ll be sure to send you some adversarial money. Finally, CVPR alone is perhaps quite a small subset to analyze trends on. In future, I hope to include more conferences as they come along.
Link to pseudo-bibliography
Acknowledgement:
A lot of times, we make grand plans for writing and publishing, only to bail out or to push it into a loop of endless procrastination. It took me 6 months of random musing to finally pen down something concrete. This could have easily stretched into, and beyond, the next CVPR, were it not for discussion with Kevin Connolly on writing and information sharing. A big shoutout to Kevin!
I am also grateful to my sister Aishwarya, Tathagata and Kevin for the amazing editorial feedback!
Thanks to Gannanath Misra for suggesting subtitle revision.






