Differentially-Private Distributed Machine Learning with Partial Worker Attendance: A Flexible and Efficient Approach

Abstract

In distributed machine learning, multiple machines or workers collaborate to train a model. However, prior research in cross-silo distributed learning with differential privacy has the drawback of requiring all workers to participate in each training iteration, hindering flexibility and efficiency. To overcome these limitations, we introduce a new algorithm that allows partial worker attendance in the training process, reducing communication costs by over 50% while preserving accuracy on benchmark data. The privacy of the workers is also improved because less data are exchanged between workers.