An Application of MapReduce Technique over Peer-to-Peer Network

P2P 네트워크상에서 MapReduce 기법 활용

  • 임건길 (동아대학교 컴퓨터공학과) ;
  • 이재기 (동아대학교 컴퓨터공학과)
  • Published : 2009.08.15

Abstract

The objective of this paper describes the design of MapReduce over Peer-to-Peer network for dynamic environments applications. MapReduce is a software framework used for Cloud Computing which processing large data sets in a highly-parallel way. Based on the Peer-to-Peer network character which node failures will happen anytime, we focus on using a DHT routing protocol which named Pastry to handle the problem of node failures. Our results are very promising and indicate that the framework could have a wide application in P2P network systems while maintaining good computational efficiency and scalability. We believe that, P2P networks and parallel computing emerge as very hot research and development topics in industry and academia for many years to come.

References

  1. J.Dean and S.Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," In Pro-ceedings of OSDI'04: Sixth Symposium in Ope-rating System Design and Implementation, pp.137-150, Dec. 2004
  2. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Communi-cations of the ACM, vol.51, no.1, pp.107-113, Jan. 2008 https://doi.org/10.1145/1327452.1327492
  3. Hadoop http://hadoop.apache.org/core/
  4. Pastry http://freepastry.org/
  5. K. Calvert, M. Doar and E. Zegura, "Modeling In-ternet Topology," IEEE Communication Magazine, vol.35, no.6, pp.160-163, Jun. 1997 https://doi.org/10.1109/35.587723