AI Safety, Alignment & Interpretability
Gradients averaged over training batches can still expose individual images, researchers report

Researchers at Nvidia describe a method called GradInversion that recovers individual images from larger training batches. The work appeared in 2021 in the proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. A gradient is the averaged signal used to update a machine learning model during training. In joint, collaborative, and federated learning, training systems may share gradients instead of raw data. Researchers had presumed that averaging gradients over a batch kept individual examples private. The new result weakens that presumption.
What does GradInversion do?
GradInversion starts with random noise and turns it into natural images while matching the gradients. A fidelity term keeps the recovered image looking natural. The method also recovers the target class label from the gradients. In the group consistency step, several attempts that begin from different random seeds work together to improve the reconstruction. The authors report that all individual images were recovered with high fidelity.
Why did researchers think larger batches were private?
Earlier work had only recovered input images under narrow conditions, such as a single input point, a network without non-linearities, the mathematical functions that let networks learn complex relationships, or a small 32 by 32 pixel batch. Because gradients are averaged across a batch, researchers thought larger batches were safe. GradInversion recovered images from batches of 8 to 48 images on a 50-layer ResNet and the ImageNet dataset, which has 1,000 classes and images of 224 by 224 pixels. That result challenges the earlier safety assumption in the studied setting.
What does this mean for privacy in federated learning?
Federated and collaborative learning systems often share gradients rather than raw training data. This paper indicates that gradients can carry enough information to identify individual images. The finding comes from a laboratory benchmark. The paper reports recovery under the conditions it studied. It does not describe a test on a live federated-learning deployment.
FAQ
What is a gradient?
A gradient is the signal that a training batch produces and a network uses to update its parameters. It is averaged across the examples in the batch.
What batch sizes and data did the researchers use?
The paper reports recovery for batches of 8 to 48 images. The tests used a 50-layer ResNet and the ImageNet dataset, which has 1,000 classes and images of 224 by 224 pixels.
Limits and what's next
The paper reports results for a 50-layer ResNet trained on ImageNet, with images of 224 by 224 pixels and batches of 8 to 48 images. The method recovers images and target class labels in that benchmark setting. This is one study, and the abstract reports no independent replication. The reported findings cover the conditions described in the abstract. They do not show how broadly the method works on other datasets, batch sizes, or network designs.
Source: Hongxu Yin et al., “See through Gradients: Image Batch Recovery via GradInversion,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings, 2021.