User-level parallel I/O configuration optimize strategy toward large-scale cluster
Author:
Affiliation:

(1. Institute of Applied Physics and Computational Mathematics, Beijing 100094, China;2. College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China)

Clc Number:

TP316

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Three key factors exert big influence upon the application′s I/O performance, including the I/O programming interface, the performance characteristic of I/O sub-system( both architecture and system software), and the I/O configuration parameters at user-level. From the user′s perspective, this paper discussed the user-level parallel I/O configuration optimize space toward large scale cluster. Besides, we proposed a method of testing and analyzing the I/O characteristic of large scale cluster. Based on this method, the I/O performance portrait of a domestic super computer was built up and several user-level parallel I/O optimize suggestions were put forward. With these carefully selected I/O configuration parameters, the time of restart data write operation was cut down by 15 percent under 8192 processes in a real application environment, while the program′s initial time is shortened from 10 minutes to 5 seconds at the scale of 4096 processes.

    Reference
    Related
    Cited by
Get Citation
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:September 19,2019
  • Revised:
  • Adopted:
  • Online: April 29,2020
  • Published: April 28,2020
Article QR Code