A new methodology for edge intelligence data quality evaluation in IDD abd Non-IID datasets in federated learning

Abstract

Massive data generation from Internet of Things (IoT) devices increases the demand for efficient data analysis to extract meaningful insights. Federated Learning (FL) allows IoT devices to collaborate in Artificial Intelligence (AI) training models while preserving data privacy. However, selecting high-quality data for training remains a critical challenge in FL environments with non-independent and identically distributed (non-iid) data. Poor-quality data introduce errors, delay convergence, and increase computational costs.

This study develops a data quality analysis algorithm for FL and centralized environments to address these challenges. The proposed algorithm reduces computational costs, eliminates unnecessary data processing, and accelerates AI model convergence.

The experiments used the MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets, and performance evaluation was based on main literature metrics like accuracy, recall, F1 score, and precision. Results show the best-case execution time reductions of up to 56.49%, with an accuracy loss of around 0.50%.