引用本文: | 胡亚红,吴寅超,朱正东.节点实时性能自适应的集群资源分配算法[J].国防科技大学学报,2022,44(6):144-150.[点击复制] |
HU Yahong,WU Yinchao,ZHU Zhengdong.Node real-time performance adaptive cluster resource scheduling algorithm[J].Journal of National University of Defense Technology,2022,44(6):144-150[点击复制] |
|
|
|
本文已被:浏览 3308次 下载 2474次 |
节点实时性能自适应的集群资源分配算法 |
胡亚红1,吴寅超1,朱正东2 |
(1. 浙江工业大学 计算机科学与技术学院(软件学院), 浙江 杭州 310023;2. 西安交通大学 计算机科学与技术学院, 陕西 西安 710049)
|
摘要: |
由于配置和所运行作业的不同,集群各节点的实时性能差异较大。为提高集群性能,提出节点实时性能自适应的集群资源分配算法 (node real-time performance adaptive cluster resource scheduling algorithm,NPARSA)。节点实时性能用其配置(CPU核数及速度、内存容量、磁盘容量)和实时状态参数(CPU、内存和磁盘的剩余数量及磁盘读写速度)表示。NPARSA根据作业类型自主选择节点性能评价指标的权值,实现节点实时性能对于作业类型的自适应。实时性能最优的节点分配给作业。虚拟机实验和物理集群实验表明,与Spark默认资源分配算法、没有考虑作业类型与节点匹配的算法、使用作业和节点匹配差异程度作为资源分配依据的算法相比,NPARSA能更有效地缩短作业执行时间、提高集群性能。 |
关键词: 资源分配 作业类型 节点实时性能 层次分析法 |
DOI:10.11887/j.cn.202206018 |
投稿日期:2021-01-08 |
基金项目:国家重点研发计划资助项目(2018YFB0204003) |
|
Node real-time performance adaptive cluster resource scheduling algorithm |
HU Yahong1, WU Yinchao1, ZHU Zhengdong2 |
(1. College of Computer Science and Technology(College of Software), Zhejiang University of Technology, Hangzhou 310023, China;2. School of Computer Science and Technology, Xi′an Jiaotong University, Xi′an 710049, China)
|
Abstract: |
The real-time performance of each node in one cluster varies greatly due to different configurations and the job running on it. To improve the cluster performance, NPARSA (node real-time performance adaptive cluster resource scheduling algorithm) was proposed. The real-time performance of a cluster node was represented by its configuration (such as the number of its CPU cores,the speed of CPU, memory capacity, and disk capacity) and the real-time state parameters (such as the residual of CPU, memory, and disk). NPARSA chose the attribute weights for a node according to the type of the job to be handled, and assigned nodes with higher priority to the job. Virtual machine experiments and physical cluster experiments prove the effectiveness of NPARSA. Compared with Spark′s default scheduling algorithm,the algorithm that does not consider the job type and node matching, and the algorithm that uses the degree of job and node matching difference as the basis for resource allocation, NPARSA can improve the performance of a cluster and shorten the execution time of user jobs. |
Keywords: resource assignment job type node real-time performance analytic hierarchy process |
|
|
|
|
|