DOI QR코드

DOI QR Code

위성 데이터 분산처리 시스템 최적화 및 안정화

Optimization and Stabilization of Satellite Data Distributed Processing System

  • Choi, Yun-Soo (Korea Institute of Science & Technology Information) ;
  • Lee, Won-Goo (Korea Institute of Science & Technology Information) ;
  • Lee, Min-Ho (Korea Institute of Science & Technology Information) ;
  • Kim, Sun-Tae (Korea Institute of Science & Technology Information) ;
  • Lee, Sang-Hwan (Korea Institute of Science & Technology Information)
  • 투고 : 2013.09.14
  • 심사 : 2013.10.08
  • 발행 : 2013.11.29

초록

본 논문은 다양한 분야에서 활용되고 있는 초기의 위성 데이터가 자체적으로 내포하고 있는 많은 왜곡 요소(구름, 광휘 등)에 대한 보정 작업을 클러스터에서 분산 처리함으로써 성능 향상 및 안정성을 제공해 주는 것이 목적이다. 이를 위해 로컬 스토리지와 상태 테이블을 이용한SGE(Sun Grid Engine) 기반 분산 처리 방식을 제안하였고, 시스템으로 구현하였다. 이를 검증하기 위해 7개의 실행노드를 이용한 실험 결과에서는 기존 시스템보다 138.81%의 성능 향상을 가져옴을 알 수 있었으며, 분산 처리 작업에 대한 안정성 또한 확보됨을 보여주었다. 이는 위성 데이터에 대한 분산 처리 작업이 I/O바운드 작업보다는 CPU 바운드 작업에 보다 더 적합하다는 것을 보여주며, 이러한 SGE 기반 분산 처리 방식은 위성영상 데이터를 처리해야하는 다양한 분야에서의 데이터 처리 및 분석 속도 향상을 가져올 수 있고, 더 나아가 근실시간적 서비스를 가능하게 할 것으로 기대한다.

The goal of this paper is to provide performance improvement and stability for satellite data correction of some distortions due to cloud or radiance through distributed processing on cluster. To do this, we proposed and implemented SGE(Sun Grid Engine) based distributed processing methods using local storages and a status table. In the verification, the experiment result revealed that the proposed system on seven nodes improved the processing speed by 138.81% as compare to the existing system and provided good stability as well. This result showed that the proposed distributed processing work is more appropriate to process CPU bound jobs than I/O bound jobs. We expect that the proposed system will give scientists improved analysis performance in various fields and near-real time analysis services.

키워드

참고문헌

  1. M. A. Mustapha, S. Sei-Ichi, T. Lihan, "Satellite-measured seasonal variations in primary production in the scallop-farming region of the Okhotsk Sea," ICES Journal of Marine Science, Vol. 66, No. 7, pp. 1557-1569, April 2009. https://doi.org/10.1093/icesjms/fsp142
  2. Adam Keith, Steve Bochinger, Euroconsult, "Satellite-based Earth Observation: Market Prospects to 2018" Euroconsult, 2009.
  3. OBPG, "SeaDAS Training Manual," http://seadas.gsfc.nasa.gov/SeaDAS_Training
  4. OBPG, "Ancillary Files for Level 1B and Level 2," http://seadas.gsfc.nasa.gov/doc/toplevel/ancinfo.html
  5. Sakharin Suwannathatsa, Prungchan Wongwises, "Chlorophyll distribution by oceanic model and satellite data in the Bay of Bengal and Andaman Sea," Oceanological and Hydrobiological Studies, Vol. 42, No. 2, pp. 132-138, June 2013. https://doi.org/10.2478/s13545-013-0066-y
  6. "Grid Schduler/Grid Engine HOWTOs," http://gridscheduler.sourceforge.net/howto/GridEngine Howto.html
  7. Amit Sheth, "A new landscape for distributed and parallel data management," Distributed and Parallel Databases, Vol. 30, No. 2, pp. 101-103 , April 2012. https://doi.org/10.1007/s10619-012-7091-5
  8. W. Gentzsch, "Sun Grid Engine: towards creating a computing power grid," Proceedings 1st IEEE/ACM Intl. Symp. on CC&G, pp. 35-36, May 2001.
  9. Akihiro Nakamura, Jong Geol Park, Kotaro Matsushita, Kenneth J. Mackin, Eiji Nunohiro, "Development and evaluation of satellite image data analysis infrastructure," Artificial Life and Robotics, Vol. 16, No. 4, pp. 511-513, Feb. 2012. https://doi.org/10.1007/s10015-011-0973-1
  10. El-Sayed M. T. El-kenawy, Ali Ibraheem El-Desoky, Mohamed F. Al-rahamawy, "Distributing Graphic Rendering using Grid Computing with Load Balancing," International Journal of Computer Applications, Vol. 47, No. 9, pp. 1-6, June 2012.
  11. Yunsoo Choi, Minho Lee, SangHwan Lee, "Evaluating the Scalability of Distributed Satellite Data Processing System," Proceeding of KSCI, Vol. 21, No. 2, pp. 395-397, July 2013.