| 引用本文: | 周伟,李艺颖,陈曙晖,等.模型不可知的联合相互学习.[J].国防科技大学学报,2023,45(3):118-126.[点击复制] |
| ZHOU Wei,LI Yiying,CHEN Shuhui,et al.Model agnostic federated mutual learning[J].Journal of National University of Defense Technology,2023,45(3):118-126[点击复制] |
|
| |
|
|
| 本文已被：浏览 4718次 下载 2960次 |
| 模型不可知的联合相互学习 |
| 周伟,李艺颖,陈曙晖,丁博 |
|
(国防科技大学 计算机学院, 湖南 长沙 410073)
|
| 摘要: |
| 主流的联邦学习(federated learning, FL)方法需要梯度的交互和数据同分布的理想假定,这就带来了额外的通信开销、隐私泄露和数据低效性的问题。因此,提出了一种新的FL框架,称为模型不可知的联合相互学习 (model agnostic federated mutual learning, MAFML)。MAFML仅利用少量低维的信息(例如,图像分类任务中神经网络输出的软标签)共享实现跨机构间的“互学互教”,且MAFML不需要共享一个全局模型,机构用户可以自定制私有模型。同时,MAFML使用简洁的梯度冲突避免方法使每个参与者在不降低自身域数据性能的前提下,能够很好地泛化到其他域的数据。在多个跨域数据集上的实验表明,MAFML可以为面临“竞争与合作”困境的联盟企业提供一种有前景的解决方法。 |
| 关键词: 联邦学习 域偏移 模型不可知 |
| DOI：10.11887/j.cn.202303014 |
| 投稿日期：2021-06-30 |
| 基金项目:科技创新2030-“新一代人工智能”重大资助项目(2020AAA0104803) |
|
| Model agnostic federated mutual learning |
| ZHOU Wei, LI Yiying, CHEN Shuhui, DIND Bo |
|
(College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China)
|
| Abstract: |
| The mainstream FL(federated learning) methods require gradient interaction and the ideal assumption of the independently identically distribution, which brings additional communication overhead, privacy leakage, and data inefficiency. Therefore, a new FL framework called MAFML (model agnostic federated mutual learning) was proposed. MAFML only used a small amount of low-dimensional information (for example, the soft labels output of the neural network in the image classification task) for sharing to achieve cross-participants “mutual learning and mutual education”. Moreover, MAFML did not need a shared global model, users can customize their own private models without restricting the model structure and parameters. At the same time, MAFML used a general approach for avoiding gradient interference so that each participant′s model could be well generalized to other domains without reducing the performance of its own domain data. Experiments on multiple cross-domain datasets show that MAFML can provide a promising solution for alliance business facing the “competition and cooperation” dilemma. |
| Keywords: federated learning domain shift model agnostic |
|
|
|
|
|
