节点实时性能自适应的集群资源分配算法
作者:
作者单位:

(1. 浙江工业大学 计算机科学与技术学院(软件学院), 浙江 杭州 310023;2. 西安交通大学 计算机科学与技术学院, 陕西 西安 710049)

作者简介:

胡亚红(1971—),女,陕西西安人,副教授,博士,硕士生导师,E-mail:huyahong@zjut.edu.cn

通讯作者:

中图分类号:

TP393

基金项目:

国家重点研发计划资助项目(2018YFB0204003)


Node real-time performance adaptive cluster resource scheduling algorithm
Author:
Affiliation:

(1. College of Computer Science and Technology(College of Software), Zhejiang University of Technology, Hangzhou 310023, China;2. School of Computer Science and Technology, Xi′an Jiaotong University, Xi′an 710049, China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    由于配置和所运行作业的不同,集群各节点的实时性能差异较大。为提高集群性能,提出节点实时性能自适应的集群资源分配算法 (node real-time performance adaptive cluster resource scheduling algorithm,NPARSA)。节点实时性能用其配置(CPU核数及速度、内存容量、磁盘容量)和实时状态参数(CPU、内存和磁盘的剩余数量及磁盘读写速度)表示。NPARSA根据作业类型自主选择节点性能评价指标的权值,实现节点实时性能对于作业类型的自适应。实时性能最优的节点分配给作业。虚拟机实验和物理集群实验表明,与Spark默认资源分配算法、没有考虑作业类型与节点匹配的算法、使用作业和节点匹配差异程度作为资源分配依据的算法相比,NPARSA能更有效地缩短作业执行时间、提高集群性能。

    Abstract:

    The real-time performance of each node in one cluster varies greatly due to different configurations and the job running on it. To improve the cluster performance, NPARSA (node real-time performance adaptive cluster resource scheduling algorithm) was proposed. The real-time performance of a cluster node was represented by its configuration (such as the number of its CPU cores,the speed of CPU, memory capacity, and disk capacity) and the real-time state parameters (such as the residual of CPU, memory, and disk). NPARSA chose the attribute weights for a node according to the type of the job to be handled, and assigned nodes with higher priority to the job. Virtual machine experiments and physical cluster experiments prove the effectiveness of NPARSA. Compared with Spark′s default scheduling algorithm,the algorithm that does not consider the job type and node matching, and the algorithm that uses the degree of job and node matching difference as the basis for resource allocation, NPARSA can improve the performance of a cluster and shorten the execution time of user jobs.

    参考文献
    相似文献
    引证文献
引用本文

胡亚红,吴寅超,朱正东.节点实时性能自适应的集群资源分配算法[J].国防科技大学学报,2022,44(6):144-150.
HU Yahong, WU Yinchao, ZHU Zhengdong. Node real-time performance adaptive cluster resource scheduling algorithm[J]. Journal of National University of Defense Technology,2022,44(6):144-150.

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-01-08
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2022-12-01
  • 出版日期: 2022-12-28
文章二维码