引用本文: | 田鸿运,刘旭,武林平,等.基于作业记账日志的并行作业特征分析工具 JobCAT[J].国防科技大学学报,2022,44(5):63-70.[点击复制] |
TIAN Hongyun,LIU Xu,WU Linping,et al.Parallel job characteristic analysis toolkit based on job accounting logs:JobCAT[J].Journal of National University of Defense Technology,2022,44(5):63-70[点击复制] |
|
|
|
本文已被:浏览 4268次 下载 2772次 |
基于作业记账日志的并行作业特征分析工具 JobCAT |
田鸿运1,刘旭1,武林平1,罗红兵1,莫则尧2 |
(1. 北京应用物理与计算数学研究所, 北京 100094;2. 中物院高性能数值模拟软件中心, 北京 100088)
|
摘要: |
并行作业特征分析是负载分析的重要基础。作业记账日志是开展作业特征分析的重要数据源。由于作业记账日志中没有记录应用名称,现有工具无法按应用名称开展作业特征分析。提出基于关键字模糊匹配的作业记账日志标记方法,设计通用的作业数据模型和柔性可扩展软件架构,集成实现并行作业特征分析工具JobCAT。通过某超级计算机系统百万量级作业记账日志数据测试验证,JobCAT的作业记账日志标记率大于95%。JobCAT支持7个插件、29项统计报表,可一键生成应用的作业特征分析报告,对负载分析研究具有实用价值。 |
关键词: 负载分析 作业记账日志 作业特征分析工具 超级计算机 |
DOI:10.11887/j.cn.202205007 |
投稿日期:2021-12-29 |
基金项目:国家重点研发计划资助项目(2018YFB0204003) |
|
Parallel job characteristic analysis toolkit based on job accounting logs:JobCAT |
TIAN Hongyun1, LIU Xu1, WU Linping1, LUO Hongbing1, MO Zeyao2 |
(1. Institute of Applied Physics and Computational Mathematics, Beijing 100094, China;2. CAEP Software Center for High Performance Numerical Simulation, Beijing 100088, China)
|
Abstract: |
Characteristic analysis of parallel job is the basics of workload analysis research. Job accounting log is an important data source of job characteristic analysis, however, existing tools cannot do statistics analysis with application name, because of application name not recorded in job accounting log. To solve this problem, a novel marking method for job accounting log was proposed, which based on keyword fuzzy matching. Combined with a general job data model and a flexible extensible software architecture, a parallel job feature analysis tool JobCAT was implemented. According to verification test by millions of job accounting log data from a supercomputer system, the log marking rate of JobCAT was greater than 95%. JobCAT supports 7 plugins and 29 statistical reports, and can easily make analysis report classified by application name, which has practical value to workload analysis research. |
Keywords: workload analysis job accounting log job characteristic analysis toolkit super computer |
|
|
|
|
|