Machine Learning Methods
Uneven data across silos weakened federated learning accuracy, and no existing method outperformed all others in every setting
In a 2022 paper, researchers from the National University of Singapore and Shanghai Jiao Tong University tested federated learning under uneven data conditions. Federated learning is a way for several parties to train a shared machine learning model without exchanging their raw data. Non-IID data means the parties do not hold independently and identically distributed data, so each silo can contain a different mix of examples. The researchers found that this uneven distribution significantly challenged the learning accuracy of federated learning methods. No existing method outperformed all the others across every non-IID setting they tested. The 2022 IEEE 38th International Conference on Data Engineering (ICDE) published the paper in its peer-reviewed proceedings.
Why uneven data matters
Privacy concerns and data regulations have fragmented training data across organizations and countries. A model trained under those constraints must learn from separate data silos without pulling the raw data together. The abstract notes that federated learning has drawn interest as a solution. But real-world silos rarely hold the same distribution of data. Tests that split data in rigid ways may miss cases that matter in practice.
What the researchers did
The researchers introduced data partitioning strategies meant to cover typical non-IID cases. They then ran a broad set of experiments to compare existing federated learning algorithms. The abstract says earlier studies divided data among parties in rigid ways that did not represent the range of non-IID conditions. The new partitioning strategies are meant to give future studies a more systematic way to test federated learning on uneven data.
What the study found
Non-IID data brought significant challenges to the learning accuracy of federated learning algorithms. None of the existing federated learning algorithms outperformed all the others in all cases. The authors frame these results as insights for future studies that address data-silo challenges.
FAQ
What is federated learning?
Federated learning is a way for multiple parties to train a shared machine learning model without exchanging their local data.
What does non-IID data mean?
Non-IID data means the parties do not hold independently and identically distributed data. Each silo can contain a different mix of examples from the others.
Which federated learning algorithm is best for non-IID data?
No single existing method outperformed all the others across every non-IID setting the researchers tested.
Limits and what's next
The abstract reports one experimental study of federated learning under non-IID data conditions. It does not name the datasets used, the number of parties involved, or the exact machine learning tasks. It reports no independent replication of the findings. The abstract presents the results as insights for future studies that work on data-silo challenges. The results leave open which method to choose for any particular non-IID data setting.
Source: Qinbin Li et al., “Federated Learning on Non-IID Data Silos: An Experimental Study,” 2022 IEEE 38th International Conference on Data Engineering (ICDE), 2022.