Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Comparative Analysis of Container for High Performance Computing

  • Lee, Jaeryun (School of Computer Science and Engineering, Kyungpook National University) ;
  • Chae, Yunchang (School of Computer Science and Engineering, Kyungpook National University) ;
  • Tak, Byungchul (School of Computer Science and Engineering, Kyungpook National University)
  • Received : 2020.08.03
  • Accepted : 2020.09.11
  • Published : 2020.09.29
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose the possibility of using containers in the HPC ecosystem and the criteria for selecting a proper PMI library. Although demand for container has been growing rapidly in the HPC ecosystem, Docker container which is the most widely used has a potential security problem and is not suitable for the HPC. Therefore, several HPC containers have appeared to solve this problem and the chance of performance differences also emerged. For this reason, we measured the performance difference between each HPC container and Docker container through NAS Parallel Benchmark experiment and checked the effect of the type of PMI library. As a result, the HPC container and the Docker container showed almost the same performance as native, or in some cases, rather better performance was observed. In the result of comparison between PMI libraries showed that PMIx was not superior to PMI-2 in all conditions.

이 논문에서는 HPC 환경에서 컨테이너의 사용 가능성을 탐구하고 적절한 PMI 라이브러리를 선택하는 기준을 실험을 통해 제안한다. HPC 환경에서 컨테이너 기술의 필요성이 대두되고 있지만 Docker 컨테이너 경우 잠재적인 보안 문제가 있어 HPC 환경에 적합하지 않다는 문제가 있다. 이러한 문제를 해결하기 위해 여러 HPC 컨테이너들이 등장하였지만, 서로 다른 특징들 따른 성능 차이가 발생할 수 있다. 그리하여 본 논문에서는 각 HPC 컨테이너 및 Docker 컨테이너 사이의 성능 차이를 NAS Parallel Benchmark 실험을 통해 측정하고 PMI 라이브러리의 종류에 따른 성능 변화를 확인해보았다. 그 결과 HPC 컨테이너와 Docker 컨테이너는 거의 네이티브와 유사한 성능을 보이거나 경우에 따라 오히려 나은 성능이 관측되었다. 또한 PMI 라이브러리 간의 비교에서는 PMIx가 PMI-2에 비해 모든 조건에서 뛰어난 것은 아닌 것으로 관측된 것을 확인할 수 있었다.

Keywords

References

  1. D.M. Jacobsen and R.S. Canon. "Contain this, unleashing docker for hpc," Proceedings of the Cray User Group, pp. 33-49, Apr. 2015.
  2. G.M. Kurtzer, V. Sochat and M.W. Bauer. "Singularity: Scientific containers for mobility of compute," PloS one, Vol. 12, No. 5, e0177459, May. 2017. DOI: 10.1371/journal.pone.0177459
  3. R. Priedhorsky and T. Randles, "Charliecloud: unprivileged containers for user-defined software stacks in HPC," Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2017, pp. 1-10, Nov. 2017. DOI: 10.1145/3126908.3126925
  4. L. Gerhardt, W. Bhimji, S. Cannon, M. Fasel, D. Jacobsen, M. Mustafa, J. Porter and V. Tsulaia, "Shifter: Containers for HPC," Journal of Physics: Conference Series, Vol. 898, No. 8, pp. 082021, Nov. 2017. DOI: 10.1088/1742-6596/898/8/082021
  5. P. Balaji, D. Buntians, D. Goodell, W. Gropp, J. Krishna, E. Lusk and R. Thakur, "PMI: A scalable parallel process-management interface for extreme-scale systems," European MPI Users' Group Meeting, pp. 31-41, Nov. 2010. DOI: 10.1007/978-3-642-15646-5_4
  6. R.H. Castain, J. Hursey, A. Bouteille and D. Solt , "PMIx: process management for exascale environments," Parallel Computing, Vol. 79, pp. 9-29, Nov. 2018. DOI: 10.1016/j.parco.2018.08.002
  7. S. Chakraborty, H. Subramoni, J. Perkins and D.K. Panda, "SHMEMPMI -- Shared Memory Based PMI for Improved Performance and Scalability," 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp.60-69, May. 2016 DOI: 10.1109/CCGrid.2016.99
  8. D.H. Bailey, E. Barszcz, J.T. Barton, D.S. Browning, R.L. Carter, L. Dagum, R.A. Fatoohi, P.O. Frederickson, T.A. Lasinski, R.S. Schreiber, H.D. Simon, V. Venkatakrishnan and S.K. Weeratunga, "The nas parallel benchmarks," International Journal of High Performance Computing Applications, VOL. 5, No. 3, pp. 63-73, Sep. 1991.
  9. S. Soltesz, H. Potzl, M.E. Fiuczynski A., Bavier, and L. Peterson, "Container-based operating system virtualization: A scalable, high-performance alternative to hypervisors," ACM SIGOPS Operating Systems Review, p.p. 275-287, Mar. 2007. DOI: 10.1145/1272996.1273025
  10. M.G. Xavier, M.V. Neves, F.D. Rossi, T.C. Ferreto, T. Lange and C.A.F De Rose, "Performance Evaluation of Container-Based Virtualization for High Performance Computing Environments," 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, p.p. 233-240, February 2013. DOI: 10.1109/PDP.2013.41
  11. W. Felter, A. Ferreira, R. Rajamony and J. Rubio, "An updated performance comparison of virtual machines and linux containers," 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), p.p. 171-172, March 2015. DOI:10.1109/ISPASS.2015.7095802
  12. M. de Bayser, and R. Cerqueira, "Integrating MPI with Docker for HPC", 2017 IEEE International Conference on Cloud Engineering (IC2E), p.p. 259-265, April 2017. DOI:10.1109/IC2E.2017.40
  13. A. Azab, "Enabling Docker Containers for High-Performance and Many-Task Computing," 2017 IEEE International Conference on Cloud Engineering (IC2E), p.p. 279-285, April 2017. DOI:10.1109/IC2E.2017.52
  14. J. Sparks, "Enabling Docker for HPC," Concurrency and computation, Vol. 31, No. 16, e5018, July 2018. DOI: 10.1002/cpe.5018
  15. A.J. Younge, K. Pedretti, R.E. Grant and R. Brightwell, "A Tale of Two Systems Using Containers to Deploy HPC Applications on Supercomputers and Clouds," 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), p.p. 74-81, Dec. 2017. DOI: 10.1109/CloudCom.2017.40
  16. O. Rudyy, M. Garcia-Gasulla, F. Mantovani, A. Santiago, R. Sirvent, and M. Vazquez, "Containers in hpc: A scalability and portability study in production biological simulations," 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS), p.p. 567-577, May 2019. DOI: 10.1109/IPDPS.2019.00066
  17. A. Torrez, T. Randles and R. Priedhorsky, "HPC container runtimes have minimal or no performance impact," 2019 IEEE/ACM International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC), pp.37-42, Nov. 2019 . DOI: 10.1109/CANOPIE-HPC49598.2019.00010