Machine Learning Methods
A 2026 study finds that variance in machine learning cross-validation rises as the number of folds grows, while bias depends on the model and dataset

In a 2026 paper in Scientific Reports, researchers at The University of Sydney tested how the number of folds in k-fold cross-validation relates to bias and variance in supervised machine learning. K-fold cross-validation is a common statistical method for estimating how well a model will perform on new data. It splits the data into k parts, trains on some parts and tests on the rest. The team varied k from 3 to 20 across four classification algorithms: support vector machine, decision tree, logistic regression and k-nearest neighbors. They evaluated the algorithms on twelve datasets of different sizes. The researchers defined bias as the difference between mean cross-validated training accuracy and held-out test accuracy. They defined variance as how much accuracy changed from fold to fold.
Across all algorithms and datasets considered, variance increased as k grew. The authors interpret this as a sign that larger k values can yield less stable fold-to-fold estimates in their setting. Bias did not follow one pattern. With increasing k, k-nearest neighbors and support vector machine most often showed upward bias, decision tree stayed comparatively balanced, and logistic regression showed mixed patterns.
How did bias and variance behave as k increased?
Variance rose with larger k across all algorithms and datasets. Bias depended on the algorithm and dataset. K-nearest neighbors and support vector machine most often showed upward bias as k increased. Decision tree stayed comparatively balanced, and logistic regression produced mixed patterns.
What does this mean for statistical validation of machine learning models?
The results suggest that common fixed choices such as 5 or 10 may not be universally optimal. The findings are limited to the models, metrics, and datasets studied. The authors provide code and data preprocessing scripts for replication and encourage further investigation into adaptive, model- and data-sensitive validation strategies.
Frequently asked questions
Why does the number of folds matter in machine learning?
The number of folds controls how the data is split for training and testing. In this study, larger k from 3 to 20 came with higher variance, meaning accuracy estimates changed more from fold to fold. Bias did not shift in one direction for all models.
Did bias rise for every algorithm?
No. K-nearest neighbors and support vector machine most often showed upward bias as k rose. Decision tree stayed comparatively balanced, and logistic regression showed mixed patterns.
Should I always use 5 or 10 folds?
The results do not show that this fixed choice is universally optimal. The authors suggest that defaults such as 5 or 10 may not be universally optimal, and they call for further work on validation strategies that adapt to the model and data.
Limits and what's next
The abstract describes one study of four classification algorithms: support vector machine, decision tree, logistic regression and k-nearest neighbors. The researchers evaluated the algorithms on twelve datasets of varying sizes, with k from 3 to 20, using accuracy as the performance metric. They defined bias as the difference between mean cross-validated training accuracy and held-out test accuracy. They defined variance as the variability of accuracy across folds. The findings are limited to the models, metrics, and datasets studied, so the patterns may not transfer to other setups. The abstract reports no independent replication or external validation. The authors encourage further investigation into adaptive, model- and data-sensitive validation strategies.
Source: Tahsinul Abedin et al., “The impact of K selection in K‑fold cross-validation on bias and variance in supervised learning models,” Scientific Reports, 2026.