AI Safety, Alignment & Interpretability
A set of natural images reduced one vision model's accuracy by about 90 percent
In a 2021 paper in the proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, researchers introduced two image datasets that reliably make machine vision models perform much worse. The examples are real, unmodified photos. The paper reports that a DenseNet-121 model scored about 2 percent accuracy on IMAGENET-A, a drop of roughly 90 percent.
What did the researchers create?
The team created IMAGENET-A, a far harder version of the ImageNet test set, and IMAGENET-O, an out-of-distribution detection dataset for ImageNet models. They used a filtering method that kept examples with limited spurious cues, patterns that can let a model take a shortcut instead of understanding the image. The examples transferred to various unseen models reliably, which the authors read as evidence that computer vision models share weaknesses.
What did the tests show?
The clearest measured result came on IMAGENET-A. A DenseNet-121 model earned around 2 percent accuracy there, an accuracy drop of approximately 90 percent. On IMAGENET-O, the same model's out-of-distribution detection was near random chance. The researchers also found that existing data augmentation techniques barely improved performance, and extra public training data gave only limited help. Improvements to computer vision architectures looked like a more promising path toward reliable models.
What else should a reader know?
What is an adversarial example?
It is an input that causes a machine learning model to make a mistake. In this work, the examples are natural, unmodified photos that still degrade model performance.
What does out-of-distribution detection mean?
It is the task of noticing inputs that differ from the data a model was trained on. IMAGENET-O tests this for ImageNet models.
Did the paper find a reliable fix?
No. The paper reports that data augmentation barely helped and extra public training data helped only a limited amount. The authors point to better model architectures as a promising direction, not a solved problem.
Limits and what's next
The abstract describes a benchmark study on two datasets, IMAGENET-A and IMAGENET-O. It gives a measured accuracy result for one model, DenseNet-121, and says the datasets transferred to various unseen models, but the abstract does not list those other models or the dataset sizes. The work is one study, and the abstract reports no independent replication. The paper also reports that data augmentation and extra public training data offered limited gains, so the architecture direction remains a line of research rather than a confirmed fix.
Source: Dan Hendrycks et al., “Natural Adversarial Examples,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings, 2021.