DOI QR코드

DOI QR Code

HTCaaS(High Throughput Computing as a Service) in Supercomputing Environment

슈퍼컴퓨팅환경에서의 대규모 계산 작업 처리 기술 연구

  • 김석규 (상명대학교 게임학과) ;
  • 김직수 (한국과학기술정보연구원 국가슈퍼컴퓨팅연구소) ;
  • 김상완 (한국과학기술정보연구원 국가슈퍼컴퓨팅연구소) ;
  • 노승우 (한국과학기술정보연구원 국가슈퍼컴퓨팅연구소) ;
  • 김서영 (한국과학기술정보연구원 국가슈퍼컴퓨팅연구소) ;
  • 황순욱 (한국과학기술정보연구원 국가슈퍼컴퓨팅연구소)
  • Received : 2014.02.04
  • Accepted : 2014.03.12
  • Published : 2014.05.28

Abstract

Petascale systems(so called supercomputers) have been mainly used for supporting communication-intensive and tightly-coupled parallel computations based on message passing interfaces such as MPI(HPC: High-Performance Computing). On the other hand, computing paradigms such as High-Throughput Computing(HTC) mainly target compute-intensive (relatively low I/O requirements) applications consisting of many loosely-coupled tasks(there is no communication needed between them). In Korea, recently emerging applications from various scientific fields such as pharmaceutical domain, high-energy physics, and nuclear physics require a very large amount of computing power that cannot be supported by a single type of computing resources. In this paper, we present our HTCaaS(High-Throughput Computing as a Service) which can leverage national distributed computing resources in Korea to support these challenging HTC applications and describe the details of our system architecture, job execution scenario and case studies of various scientific applications.

Keywords

High-Throughput Computing;Meta-Job;Multi-level Scheduling;HTCaaS

References

  1. D. Thain, T. Tannenbaum, and M. Livny, "Distributed Computing in Practice: The Condor Experience," Concurrency and Computation:Practice and Experience, Vol. 17, Issue 2-4, pp.323-356, 2005. https://doi.org/10.1002/cpe.938
  2. 김성준, 성진우, 장지훈, 이상동,"국가 슈퍼컴퓨팅 공동활용 환경을 위한 통합 모니터링 환경 구축", 한국콘텐츠학회 추계 종합학술대회 논문집, 제5권, 제2호(하), pp.517-521, 2007.
  3. 우준, 박석중, 이상동, 김형식,"국가 슈퍼컴퓨팅 공동활용체제 구축을 위한 글로벌공유파일시스템 성능 분석", 한국콘텐츠학회 추계 종합학술대회 논문집, 제5권, 제2호(하), pp509-512, 2007.
  4. http://research.cs.wisc.edu/htcondor/
  5. A. Casa jusetal,"DIRAC Pilot Framework and the DIRAC Workload Management System," Journal of Physics:Conference Series, Vol.219, No.6, p.062049, 2010. https://doi.org/10.1088/1742-6596/219/6/062049
  6. http://lhcb.web.cern.ch/lhcb/
  7. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, and Mike Wilde,"Falkon: a Fastand Light-weight tasK executiON framework," ACM/IEEE conference on Supercomputing (SC'07), 2007.
  8. I. Raicu, I. Foster, and Y. Zhao, "Many-Task Computing for Grids and Supercomputers," ACM Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS'08), 2008.
  9. www.cern.ch/alice/
  10. http://alien2.cern.ch/
  11. http://cern.ch/DIANE
  12. http://autodock.scripps.edu
  13. T. T. Nguyen, H. J. Ryu, S. H. Lee, S. Hwang, V. Breton, J. H. Rhee, and D. Kim,"Virtual screening identification of novelsevere acute respiratory syndrome 3C-like protease inhibitors and in vitro confirmation," Bioorganic & Medicinal Chemistry Letters, Vol. 21, No. 10, pp.3088-3091, 2011. https://doi.org/10.1016/j.bmcl.2011.03.034
  14. T. T. Nguyen, H. J. Ryu, S. H. Lee, S. Hwang, J. Cha, V. Breton, and D. Kim, "Discovery of novel inhibitors for human intestinal maltase: virtual screening in a WISDOM environment and in vitro evaluation," Biotechnology Letters, Vol. 33, No. 11, pp.2185-2191, 2001.
  15. http://madgraph.hep.uiuc.edu/
  16. http://pythia6.hepforge.org/
  17. G. Shainer, T. Liu, J. Layton, and J. Mora, "Scheduling Strategies for HPC as a Service (HPCaaS)," IEEE Cluster Computing and Workshops, 2009.
  18. http://www.egi.eu/
  19. http://boinc.berkeley.edu/
  20. http://www.plsi.or.kr