Abstract:Today’s advancing modern science is generating and analyzing increasing scale of datasets and makes HPC storage system facing new challenges both on architecture and software approach. In order to exploit potential benefits of emerging hybrid and hierarchy storage architecture on Milky-2 system, a I/O middleware approach named application coupled burst buffer, is introduced to make full use of the solid state disk based in-system storage resources. Application coupled burst buffer aggregates distributed in-system storage close to running tasks into single namespace during application runtime and manages it as cooperative persistent burst buffer tightly coupled with its host application. To take full advantage of cooperative burst buffer, application coupled burst buffer uses an unified shadow namespace to map application data into physical in system storage based on its real namespace of the host application. Besides that, application coupled burst buffer organizes data with locality aware layout and leverages application intent based replacement policy to fully exploit spatial and temporal locality. Furthermore, application coupled burst buffer employs concurrency aware policies to optimize data movement between different storage tiers. Evaluations on Milky-2 system show that application coupled burst buffer can improve the performance of typical data-intensive applications dramatically. It can achieve scalable burst I/O bandwidth and smooth sustained I/O bandwidth with high throughput solid state disk deployed and can be taken as an appropriate candidate for storage solution on emerging leadership supercomputer systems.