Abstract
Federated learning (FL) is a distributed learning approach in which clients collaboratively train a machine learning model without sharing data. While this distributed learning paradigm has clear benefits in terms of privacy, data heterogeneity among clients strengthens the local client drift causing the degradation of global model’s performance. Since gradient distances can be used to quantify the local client drift, we leverage them in the aggregation strategy to achieve robust performance in highly heterogeneous environments. In this work, we propose two adaptive aggregation algorithms that optimize the aggregation weights based on cross-rounds and cross-clients gradient distances, dubbed CROG and CLIG, respectively. Through extensive experiments using various datasets, we verify that the proposed algorithms outperform conventional FL methods, confirming that the proposed aggregation weights are formed close to the optimal aggregation weights and operate robustly in extreme non-IID data environments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
No datasets were generated or analysed during the current study.
References
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Singh, A., Zhu, J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, pp. 1273–1282 (2017). PMLR
Wang, J., Liu, Q., Liang, H., Joshi, G., Poor, H.V.: Tackling the objective inconsistency problem in heterogeneous federated optimization. Adv. Neural. Inf. Process. Syst. 33, 7611–7623 (2020)
Hsu, T.-M.H., Qi, H., Brown, M.: Measuring the effects of non-identical data distribution for federated visual classification, (2019). arXiv preprint arXiv:1909.06335
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., Chandra, V.: Federated learning with non-iid data, (2018). arXiv preprint arXiv:1806.00582
Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. Proc. Mach. learning syst. 2, 429–450 (2020)
Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S., Stich, S., Suresh, A.T.: Scaffold: Stochastic controlled averaging for federated learning. In: International Conference on Machine Learning, pp. 5132–5143. (2020). (PMLR)
Reyes, J., Di Jorio, L., Low-Kam, C., Kersten-Oertel, M.: Precision-weighted federated learning, (2021). arXiv preprint arXiv:2107.09627
Zeng, Y., Mu, Y., Yuan, J., Teng, S., Zhang, J., Wan, J., Ren, Y., Zhang, Y.: Adaptive federated learning with non-iid data. Comput. J. 66(11), 2758–2772 (2023)
Li, Z., Lin, T., Shang, X., Wu, C.: Revisiting weighted aggregation in federated learning with neural networks. In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J. (eds.) Proceedings of the 40th International Conference on Machine Learning, pp. 19767–19788. JMLR.org, Honolulu, Hawaii, USA (2023)
Rehman, Y.A.U., Gao, Y., Gusmao, P.P.B., Alibeigi, M., Shen, J., Lane, N.D.: L-dawa: Layer-wise divergence aware weight aggregation in federated self-supervised visual representation learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16464–16473. (2023)
Wu, H., Wang, P.: Fast-convergent federated learning with adaptive weighting. IEEE Trans. Cognitive Commun. Netw. 7(4), 1078–1088 (2021)
Acar, D.A.E., Zhao, Y., Navarro, R.M., Mattina, M., Whatmough, P.N., Saligrama, V.: Federated learning based on dynamic regularization, (2021). arXiv preprint arXiv:2111.04263
Li, Q., He, B., Song, D.: Model-contrastive federated learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10713–10722. (2021)
Xiao, C., et al.: Confusion-resistant federated learning via diffusion-based data harmonization on non-iid data. Adv. Neural. Inf. Process. Syst. 37, 137495–137520 (2024)
Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečnỳ, J., Kumar, S., McMahan, H.B.: Adaptive federated optimization. arXiv preprint arXiv:2003.00295 (2020)
Lin, T., Kong, L., Stich, S.U., Jaggi, M.: Ensemble distillation for robust model fusion in federated learning. Adv. Neural. Inf. Process. Syst. 33, 2351–2363 (2020)
Liu, X., Liu, L., Ye, F., Shen, Y., Li, X., Jiang, L., Li, J.: Fedlpa: One-shot federated learning with layer-wise posterior aggregation. Adv. Neural. Inf. Process. Syst. 37, 81510–81548 (2024)
Sahoo, P., Tripathi, A., Saha, S., Mondal, S.: FedDUAL: A dual-strategy with adaptive loss and dynamic aggregation for mitigating data heterogeneity in federated learning. Trans. Mach. Learn. Res. (2025)
Mollanejad, A., Ghaffari, A., Seyyedabbasi, A., Navin, A.H.: Fairness-based federated learning in autonomous vehicles with exponential moving average loss weighting. Cluster Comput. 28(10), 627 (2025)
Jain, S., Gupta, S., Pundir, A., Singh, S., Saxena, G.J.: FedA2W: adaptive aggregation with weighted loss proximity for non-iid federated learning. Neural Comput. Appl. 38(13), 561 (2026)
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, (2017). arXiv preprint arXiv:1708.07747
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images, Technical report, University of Toronto, Toronto, ON, Canada (2009)
Tian, P., Liao, W., Yu, W., Blasch, E.: Wscc: A weight-similarity-based client clustering approach for non-iid federated learning. IEEE Internet Things J. 9(20), 20243–20256 (2022)
Jeong, W., Yoon, J., Yang, E., Hwang, S.J.: Federated semi-supervised learning with inter-client consistency & disjoint learning. Int. Conference Learn. Representations (2020)
Li, Y., Wang, S., Chang, T.-H.: Differentially private federated stochastic primal-dual learning for internet of vehicles. IEEE Internet of Things J. 12(11), 17034–17050 (2025)
Acknowledgments
This research was supported by Seoul National University of Science and Technology.
Author information
Authors and Affiliations
Contributions
Conceptualization: J.B., B.P. and MH.K., Methodology: J.B., B.P. and P.G, Writing: J.B. and B.P., Data Analysis and investigation: J.B and B.P., Software: J.B., B.P. and P.G, Review and editing: MH.K and MC.K., Supervison: MH. K. and MC.K.
Corresponding authors
Ethics declarations
Conflict of interest
The authors have no competing interests to declare that are relevant to the content of this article.
Consent for publication
All authors have read and approved the final manuscript for submission and publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bae, J.H., Park, B., Gusmão, P.P.B.d. et al. Robust federated learning aggregation for highly heterogeneous data. Cluster Comput 29, 875 (2026). https://doi.org/10.1007/s10586-026-06716-3
Received:
Revised:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1007/s10586-026-06716-3