Byzantine-Strong Decentralized Federated Studying
Authors: Minghong Fang, Zifan Zhang, Hairi, Prashant Khanduri, Jia Liu, Songtao Lu, Yuchen Liu, Neil Gong
Summary: Federated studying (FL) permits a number of purchasers to collaboratively prepare machine studying fashions with out revealing their personal coaching knowledge. In standard FL, the system follows the server-assisted structure (server-assisted FL), the place the coaching course of is coordinated by a central server. Nonetheless, the server-assisted FL framework suffers from poor scalability on account of a communication bottleneck on the server, and belief dependency points. To handle challenges, decentralized federated studying (DFL) structure has been proposed to permit purchasers to coach fashions collaboratively in a serverless and peer-to-peer method. Nonetheless, on account of its totally decentralized nature, DFL is very susceptible to poisoning assaults, the place malicious purchasers might manipulate the system by sending carefully-crafted native fashions to their neighboring purchasers. So far, solely a restricted variety of Byzantine-robust DFL strategies have been proposed, most of that are both communication-inefficient or stay susceptible to superior poisoning assaults. On this paper, we suggest a brand new algorithm referred to as BALANCE (Byzantine-robust averaging via native similarity in decentralization) to defend in opposition to poisoning assaults in DFL. In BALANCE, every shopper leverages its personal native mannequin as a similarity reference to find out if the acquired mannequin is malicious or benign. We set up the theoretical convergence assure for BALANCE underneath poisoning assaults in each strongly convex and non-convex settings. Moreover, the convergence charge of BALANCE underneath poisoning assaults matches these of the state-of-the-art counterparts in Byzantine-free settings. Intensive experiments additionally display that BALANCE outperforms current DFL strategies and successfully defends in opposition to poisoning assaults