分布式异构集群中节点优先级调优算法
作者:
作者单位:

(浙江工业大学 计算机科学与技术学院(软件学院), 浙江 杭州 310023)

作者简介:

胡亚红(1971—),女,陕西西安人,副教授,博士,硕士生导师,E-mail:huyahong@zjut.edu.cn

通讯作者:

中图分类号:

TP393

基金项目:

国家重点研发计划资助项目(2018YFB0204003)


Node priority optimization in distributed heterogeneous clusters
Author:
Affiliation:

(College of Computer Science and Technology(College of Software), Zhejiang University of Technology, Hangzhou 310023, China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    节点优先级常用于评价异构集群中节点的性能,因此节点优先级评价指标权重的选择非常重要。采用层次分析法(analytic hierarchy process, AHP)建立了节点优先级评价指标体系,计算得到各指标的初始权重,并使用BP神经网络对初始权重进行优化。训练时,BP网络输入为集群运行中采集的节点实时资源数据,输出为节点的优先级。分析网络训练完成后得到的权重矩阵可以获得各优先级评价指标的优化权重。实验表明,基于AHP和BP的节点优先级评价模型可以更加准确地分析节点性能。相比于Spark默认算法和权重未优化的对照算法,使用调优后的节点优先级可以有效提高集群性能。运行不同工作量的相同负载时,集群平均性能分别提高了16.64%和9.76%;处理相同工作量的不同负载时,集群的平均性能分别提高了12.49%和6.54%。

    Abstract:

    Node priority is often used to evaluate the performance of heterogeneous cluster nodes, and it is of great importance to provide suitable weight for each priority evaluation index. The AHP (analytic hierarchy process) was chosen to establish the evaluation index system of node priority, and the initial weight of each index was calculated. The BP (back propagation) neural network was then used to optimize the weights obtained by using AHP. The input of the BP neural network was the node′s performance index values collected during execution of cluster, and the output was the corresponding priority of the node. After the network training, the weight matrix was obtained and used to calculate the optimized weights. The experimental results show that the cluster node priority evaluation model based on AHP and BP can evaluate the node performance more accurately. Compared with the default resource allocation algorithm of Spark and the comparison algorithm with unoptimized weights, the cluster performance is improved effectively by using the node priority optimized. When running the same kind of load with different amount of data, the average cluster performance increases by 16.64% and 9.76%, respectively; and when running different loads with the same amount of data, the average performance of the cluster increases by 12.49% and 6.54%, respectively.

    参考文献
    相似文献
    引证文献
引用本文

胡亚红,邱圆圆,毛家发.分布式异构集群中节点优先级调优算法[J].国防科技大学学报,2022,44(5):102-113.

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2020-11-22
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2022-09-28
  • 出版日期: 2022-10-28
文章二维码