Dataset distillation-based optimization for heterogeneous federated learning

Abstract Federated learning (FL) is a privacy-preserving distributed learning paradigm that allows multiple clients to collaboratively train a global model. However, the FL performance is often degraded by data heterogeneity, i.e., non-IID data distributed in multiple clients, and the cost of system communication can increase significantly. In this paper, we present FedDS, a novel FL method based on dataset distillation to handle data heterogeneity. Specifically, FedDS first works by extracting distilled data from local data and sharing distilled data to optimize data distribution for each FL client, which can reduce the impact of data heterogeneity. Second, FedDS clusters clients based on the similarity of distilled data features, and the most representative clients from each cluster are selected to participate in FL training, which can further improve performance. Extensive experiments on multiple benchmark datasets demonstrate the effectiveness of FedDS in improving the model’s performance compared to several advanced FL methods.