The Journal of Korean Institute of Communications and Information Sciences (한국통신학회논문지)

The Korean Institute of Commucations and Information Sciences (한국통신학회)
권호 표지

Development of a Distributed Web Caching Network through Consistent Hashing and Dynamic Load Balancing

  • Hwan Chang (Department of Control and Instrumentation Engineering, Chonbuk National University) ;
  • Jong Ho Park (Department of Mechatronics Engineering, Chonbuk National University) ;
  • Ju Ho Park (Department of Computer Science, Chonbuk National University) ;
  • Kil To Chong (Devision of Electronics and Information Engineering, Chonbuk National University)
  • Published : 2002.11.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper focuses on a hash-based, distributed Wet caching network that eliminates inter-cache communication. An agent program on cache servers, a mapping program on the DNS server, and other components comprised in a distributed Web caching network were modified and developed to implement a so-called "consistent" hashing. Also, a dynamic load balancing algorithm is proposed to address the load-balancing problem that is a key performance issue on distributed architectures. This algorithm effectively balances the load among cache servers by distributing the calculated amount of mapping items that have higher popularity than others. Therefore, this developed network can resolve the imbalanced load that is caused by a variable page popularity, a non-uniform distribution of a hash-based mapping, and a variation of cache servers.

Keywords

References

  1. IEEE Internet Computing v.5 A web caching primer B.D.Davison
  2. Proceedings of the 1996 USENIX Technical Conference A hierarchical Internet object cache A.Chankhunthod(et al.)
  3. IEEE/ACM Transactions on Networking v.8 Summary cache: A scalable wide-area web cache sharing protocol L.Fan(et al.)
  4. Workshop on Internet Server Performance (WISP98) A Taste fo Crispy Squid S.Gadde(et al.)
  5. Proceedings of the 4th International World Wide Web Conference Making Workd Wide Web Caching Servers Cooperate R.Malpani(et al.)
  6. Proceedings the 7th international World Wide Web Conference Performance study of a collaborative method for hierarchical caching in proxy servers P.S.Yu;E.A.MacNair
  7. Proceedings of the 8th International World Wide Web Conference Web caching with consistent hashing D.Karger(et al.)
  8. Proceedings of ACM Multimedia'97 A scalable and distributed WWW proxy system K.L.E.Law(et al.)
  9. Proceedings of the 2000 IEEE IPCCC Load balancing a cluster of Web servers: using distributed packet rewriting L.Aversa;A.Bestavros
  10. Proceedings of the 29th Annual ACM Symposium on Theory of Computing Consistent hashing and random trees : Distributed caching protocols for relieving hot spots on the World Wide Web D.Karger(et al.)
  11. Proceedings of the ACM SIGMETRICS Conference Generating representative web workloads for network and server performance evaluation P.Barford;M.E.Crovella
  12. The MD4 Message-Digest Algorithm, RFC 1320 R.Rivest
  13. Squid Web Proxy Cache D.Wessels
  14. Internet Software Consortium
  15. DNS and BIND(3rd edition) P.Albitz;C.Liu
  16. Internet Cache Protocol (ICP), version 2, RFC 2186 D.Wessels;K.Claffy