Fed-DNN-Debugger: Automatically Debugging Deep Neural Network Models in Federated Learning

Abstract

Federated learning is a distributed machine learning framework that has been widely applied in scenarios that require data privacy. To obtain a neural network model that performs well, when the model falls into a bug, existing solutions retrain it on a larger training dataset or the carefully selected samples from model diagnosis. To overcome this challenge, this paper presents Fed-DNN-Debugger, which can automatically and efficiently fix DNN models in federated learning. Fed-DNN-Debugger fixes the federated model by fixing each client model. Fed-DNN-Debugger consists of two modules for debugging a client model: nonintrusive metadata capture (NIMC) and automated neural network model debugging (ANNMD). NIMC collects the metadata with deep learning software syntax automatically. It does not insert any code for metadata collection into modeling scripts. ANNMD scores samples according to metadata and searches for high-quality samples. Models are retrained on the selected samples to repair their weights. Our experiments with popular federated models show that Fed-DNN-Debugger can improve the test accuracy by 8% by automatically fixing models.

Diagnosing Machine Learning Problems in Federated Learning Systems: A Case Study

“…One of the rare works to not focus on finding underperforming clients but on mitigating issues concerning FL models is Fed-DNN-Debugger [10]. It consists of two modules: one is responsible for nonintrusive metadata capture (NIMC), which produces data that is then used for automated neural network model debugging (ANNMD).…”