模型不可知的联合相互学习
作者:
作者单位:

(国防科技大学 计算机学院, 湖南 长沙 410073)

作者简介:

周伟(1990—),男,山西忻州人,工程师,博士,E-mail:zhouwei14@nudt.edu.cn; 丁博(通信作者),男,湖南攸县人,研究员,博士,硕士生导师,E-mail:dingbo@nudt.edu.cn

通讯作者:

中图分类号:

TP181

基金项目:

科技创新2030-“新一代人工智能”重大资助项目(2020AAA0104803)


Model agnostic federated mutual learning
Author:
Affiliation:

(College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    主流的联邦学习(federated learning, FL)方法需要梯度的交互和数据同分布的理想假定,这就带来了额外的通信开销、隐私泄露和数据低效性的问题。因此,提出了一种新的FL框架,称为模型不可知的联合相互学习 (model agnostic federated mutual learning, MAFML)。MAFML仅利用少量低维的信息(例如,图像分类任务中神经网络输出的软标签)共享实现跨机构间的“互学互教”,且MAFML不需要共享一个全局模型,机构用户可以自定制私有模型。同时,MAFML使用简洁的梯度冲突避免方法使每个参与者在不降低自身域数据性能的前提下,能够很好地泛化到其他域的数据。在多个跨域数据集上的实验表明,MAFML可以为面临“竞争与合作”困境的联盟企业提供一种有前景的解决方法。

    Abstract:

    The mainstream FL(federated learning) methods require gradient interaction and the ideal assumption of the independently identically distribution, which brings additional communication overhead, privacy leakage, and data inefficiency. Therefore, a new FL framework called MAFML (model agnostic federated mutual learning) was proposed. MAFML only used a small amount of low-dimensional information (for example, the soft labels output of the neural network in the image classification task) for sharing to achieve cross-participants “mutual learning and mutual education”. Moreover, MAFML did not need a shared global model, users can customize their own private models without restricting the model structure and parameters. At the same time, MAFML used a general approach for avoiding gradient interference so that each participant′s model could be well generalized to other domains without reducing the performance of its own domain data. Experiments on multiple cross-domain datasets show that MAFML can provide a promising solution for alliance business facing the “competition and cooperation” dilemma.

    参考文献
    相似文献
    引证文献
引用本文

周伟,李艺颖,陈曙晖,等.模型不可知的联合相互学习[J].国防科技大学学报,2023,45(3):118-126.
ZHOU Wei, LI Yiying, CHEN Shuhui, et al. Model agnostic federated mutual learning[J]. Journal of National University of Defense Technology,2023,45(3):118-126.

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-06-30
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2023-06-07
  • 出版日期: 2023-06-28
文章二维码