Machine Learning Methods
ConvNeXt V2 models improve image classification, detection, and segmentation results
In a 2023 paper published in peer-reviewed conference proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, researchers at Korea Advanced Institute of Science and Technology and New York University report a new family of visual-recognition models called ConvNeXt V2. Convolutional networks, the base of the new models, are a common type of image-recognition model. The models pair a self-supervised training method, masked autoencoders, with a layer called Global Response Normalization. The paper reports that this combination improves pure convolutional networks on three standard tests, ImageNet classification, COCO detection, and ADE20K segmentation.
Why did the combination need a new design?
The authors explain that modern ConvNets were originally designed for supervised learning with ImageNet labels. They also note that masked autoencoders, a self-supervised method that learns by reconstructing hidden parts of images, can help these models. But the paper reports that simply combining the two approaches led to weak performance. The new work co-designs the training method and the model. It adds a Global Response Normalization layer to sharpen competition among features inside the network.
How did the models perform?
The paper describes a range of ConvNeXt V2 model sizes. The smallest, the Atto model with 3.7 million parameters, reached 76.7% top-1 accuracy on ImageNet. Top-1 accuracy means the share of images where the model's first-ranked label was correct. The largest, the Huge model with 650 million parameters, reached 88.9% top-1 accuracy on ImageNet using only public training data. The abstract also reports improved performance on COCO detection and ADE20K segmentation, though it does not give exact figures for those two benchmarks.
FAQ
What is a ConvNet?
A ConvNet, or convolutional neural network, is a type of visual-recognition model that learns features directly from images. ConvNeXt is a modern family of these networks.
What is a masked autoencoder?
A masked autoencoder is a self-supervised training method. It hides parts of an image and trains the model to rebuild the hidden parts, which creates a learning signal without extra labels for that part of training.
Did the paper release the models?
Yes. The paper provides pre-trained ConvNeXt V2 models of several sizes, from the efficient 3.7 million parameter Atto model to the 650 million parameter Huge model.
Limits and what's next
The abstract describes one study that evaluates the new models on three benchmark datasets, ImageNet for classification, COCO for detection, and ADE20K for segmentation. It reports exact ImageNet accuracy for the Atto and Huge models but not exact score changes for COCO or ADE20K. The results are benchmark evaluations, not evidence from a real-world setting. The abstract does not report an independent replication, so these results stand as one benchmark study.
Source: Sanghyun Woo et al., “ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings, 2023.