IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Evaluation of Energy Management Algorithms for MapReduce System

MapReduce 시스템을 위한 에너지 관리 알고리즘의 성능평가

  • Received : 2014.01.10
  • Accepted : 2014.02.24
  • Published : 2014.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Analyzing large scale data has become an important activity for many organizations. Since MapReduce is a promising tool for processing the massive data sets, there are increasing studies to evaluate the performance of various algorithms related to MapReduce. In this paper, we first develop a simulation framework that includes MapReduce workload model, data center model, and the model of data access pattern. Then we propose two algorithms that can reduce the energy consumption of MapReduce systems. Using the simulation framework, we evaluate the performance of the proposed algorithms under different application characteristics and configurations of data centers.

Keywords

References

  1. J. Leverich, C. Kozyrakis, "On the Energy (in) Efficiency of Hadoop Clusters," ACM SIGOPS Operating Systems Review, Vol. 44, No. 1, pp.61-65, 2010. https://doi.org/10.1145/1740390.1740405
  2. Y. Chen, S. Alspaugh, D. Borthakur, R. Katz, "Energy Efficiency for Large-Scale Map-Reduce Workloads with Significant Interactive Analysis," Proceedings of ACM European Conference on Computer Systems, pp.43-56, 2012.
  3. Y. Chen, A. Ganapathi, R. Griffith, R. Katz, "The Case for Evaluating MapReduce Performance Using Workload Suites," Proceedings of IEEE International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems, pp.390-399, 2011.
  4. C.L. Abad, N. Roberts, Y. Lu, R.H. Campbell, "A Storage-Centric Analysis of MapReduce Workloads: File Popularity, Temporal Locality and Arrival Patterns," Proceedings of IEEE International Symposium on Workload Characterization, pp.100-109, 2012.
  5. W. Lang, J.M. Patel, "Energy Management for MapReduce Clusters," Proceedings of VLDB Endowment, Vol. 3, No. 1-2, pp.129-139, 2010.
  6. R.T. Kaushik, M. Bhandarkar, "GreenHDFS: Towards An Energy-Conserving Storage-Efficient, Hybrid Hadoop Compute Cluster," Proceedings of USENIX Annual Technical Conference, 2010.
  7. V.A. Patil, V. Chaudhary, "Rack Aware Scheduling in HPC Data Centers: An Energy Conservation Strategy," Proceedings of IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum, pp.814-821, 2011.
  8. S. Hammoud, M. Li, Y. Liu, N.K. Alham, Z. Liu, "MRSim: A Discrete Event Based MapReduce Simulator," Proceedings of International Conference on Fuzzy Systems and Knowledge Discovery, pp.2993-2997, 2010.
  9. H. Schwetman, "User's Guide: Article Reprints: CSIM 18-The Simulation Engine," World Wide Web electronic publication, 2009.