Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Design of Grid Workflow System Scheduler for Task Pipelining

작업 파이프라이닝을 위한 그리드 워크플로우 스케줄러 설계

  • Lee, In-Seon (Dept. of Computer Information Processing, Shingu University)
  • Received : 2010.04.13
  • Accepted : 2010.05.26
  • Published : 2010.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

The power of computational Grid resources can be utilized on users desktop by employing workflow managers. It also helps scientists to conveniently put together and run their own scientific workflows. Generally, stage-in, process and stage-out are serially executed and workflow systems help automate this process. However, as the data size is exponentially increasing and more and more scientific workflows require multiple processing steps to obtain the desired output, we argue that the data movement will possess high portion of overall running time. In this paper, we improved staging time and design a new scheduler where the system can execute concurrently as many jobs as possible. Our simulation study shows that 10% to 40% improvement in running time can be achieved through our approach.

워크플로우 관리자는 대량의 계산용 그리드 자원을 데스크탑 컴퓨터에서 개인이 편리하게 워크플로우를 만들고 수행할 수 있게 해주는 유용한 도구이다. 보통 데이터는 스테이지-인, 프로세스, 스테이지-아웃의 순서로 순차적으로 진행되며 워크플로우 시스템은 이 과정을 자동화해준다. 그러나 최근의 e-science에서는 사용되는 데이터 량이 급속하게 증가하고 있고 원하는 출력물을 얻기 위해 여러 번의 과정을 수행하면서 데이터 이동 시간이 전체 수행시간의 많은 부분을 차지하게 되어 스테이징 과정의 개선이 중요한 이슈가 되고 있다. 본 논문에서는 스테이징 과정을 개선하고, 이를 이용하여 가능한 한 많은 작업들을 동시 수행시키는 스케줄러를 설계하였다. 또한 모의실험을 통해 제안한 스케줄러의 성능이 10~40%까지 향상됨을 보였다.

Keywords

References

  1. 황선태, 심규호, "계산그리드에서 워크플로우기반의 사용자환경 설계 및 구현," 한국컴퓨터정보학회논문지, 제 10권, 제 4호, 165-171쪽, 2005년 9월.
  2. H.S.Kim, and H.Y.Yeom, "A task pipelining framework for e-science workflow systems," 3rd workflow Systems for e-Science(WSES), May 2008.
  3. I.Y.Jung, I.S. Cho, H.Y.Yeom, H.S. Kweon, and J.Lee, "HVEM DataGrid : Implementation of a Biologic Data Management System for Experiments with High Voltage Electron Microscope," Lecture Notes in Computer Science, Vol.4360, pp.175-190, 2006,
  4. Karan Bhatia, Sandeep Chandra, Kurt Mueller. "GAMA:Grid Account Management Architecture," International Conference on e-Science and Grid Computing(e-Science'05), 2005.
  5. K.Ranganathan, and I.Foster, "Decoupling computation and data scheduling in distributed data-intensive applications," International Symposium on High Performance Distributed Computing(HPDC), 2002.
  6. T.M. McPhillips, and S. Bowers, "An approach for pipelining nested collections in scientific workflows," ACM SIGMOD Record, Vol. 34, No.3, pp.12-17, 2005. https://doi.org/10.1145/1084805.1084809
  7. V.Bhat, S.Klasky, S.Atchley, M.Beck, D.McCune, and M.Parashar, "High performance threaded data streaming for large scale simulations," IEEE/ACM International Workshop on Grid Computing, 2004.
  8. J.Blower, K.Haines, and E.Llewellin, "Data streaming, workflow and firewall-friendly grid services with Styx," UK e-Science All Hands Meeting, Nov. 2005.
  9. D.Abramson, and J.Kommineni,"A flexible IO scheme for grid workflows," International Parallel & Distributed Processing Symposium(IPDPS), Arpil 2004.
  10. V.Korkhov, D.Vasyunin, A.Wibisono, A.S.Belloum, M.A. Inda, M.Roos, T.M.Breit, and L.Hertzberger, "VLAMG: Interactive data driven workflow engine for grid-enabled resources," Journal of Scientific Programming, Vol.15, No.3, pp.173-188, 2007. https://doi.org/10.1155/2007/812036
  11. A.Mandal, K.Kennedy, C.Koelbel, G.Marin, J.Mellor-Crummey, B.Liu, and L.Johnsson, "Scheduling strategies for mapping application workflows onto the grid," 14th IEEE International Symposium on High Performance Distributed Computing(HPDC), July 2005.
  12. S.Frdric, D.Frdric, and C.Henri, "From heterogeneous task scheduling to heterogeneous mixed parallel scheduling," International Euro-Par Conference. 2005.
  13. 이준동, 이무훈, 최의인, "그리드 컴퓨팅 환경에서 확장 가능한 분산 스케줄링," 한국컴퓨터정보학회논문지, 제12권, 제 6호, 1-9쪽, 2007년 12월.
  14. 박량재, 장성호, 조규철, 이종실, "계산그리드를 위한 퍼지로직 기반의 그리드 작업스케줄링 모델," 한국컴퓨터정보학회논문지, 제 12권, 제 5호, 49-56쪽, 2007년 11월.
  15. J.Yu, and R.Buyy, "A taxonomy of workflow management systems for grid computing," Journal of Grid Computing, Vol. 3, No. 3-4, pp.171-200, Sep. 2005. https://doi.org/10.1007/s10723-005-9010-8
  16. S.Ayyub, and D.Abramson, "GridRod- a dynamic runtime scheduler for grid workflows," International Supercomputing Conference(ISC) 2007, June 2007.