A Reconfigurable Load and Performance Balancing Scheme for Parallel Loops in a Clustered Computing Environment

클러스터 컴퓨팅 환경에서 병렬루프 처리를 위한 재구성 가능한 부하 및 성능 균형 방법

  • 김태형 (한양대학교 전자컴퓨터공학부)
  • Published : 2004.02.01

Abstract

Load imbalance is a serious impediment to achieving good performance in parallel processing. Global load balancing schemes cannot adequately manage to balance parallel tasks generated from a single application. Dynamic loop scheduling methods are known to be useful in balancing parallel loops on shared-memory multiprocessor machines. However, their centralized nature causes a bottleneck for the relatively small number of processors in a network of workstations because of order-of-magniture differences in communication overheads. Moreover, improvements of basis loops scheduling methods have not effectively dealt with irregularly distributed workloads in parallel loops, which commonly occur in applications for a network of workstation. In this paper, we present a new reconfigurable and decentralized balancing method for parallel loops on a network of workstations. Since our method supplements performance balancing with those tranditional load balancing methods, it minimizes the overall execution time.

References

  1. M. Satyanarayanan and E. H. Siegel. MultiRPC: A parallel remote procedure call mechanism. Technical Report CMU-CS-86-139, Carnegie-Mellon University, 1986
  2. Clemens H. Cap and Volker Strumpen. Efficient parallel computing in distributed workstation environments, Parallel Computing, Vol. 19: 1221-1234, 1993 https://doi.org/10.1016/0167-8191(93)90028-J
  3. A. S. Grimshaw, J. B. Weissman, E. A. West and Jr. E. C. Loyot. 'Metasystems:An approach combining parallel processing and heterogeneous distributed computing systems. Journal of Parallel and Distributed Computing, Vol. 21: 257-270, 1994 https://doi.org/10.1006/jpdc.1994.1058
  4. P. Krueger and N. G. Shivaratri. Adaptive location policies for global scheduling. IEEE Transactions on Software Engineering, Vol. 20(6): 432-444, June 1994 https://doi.org/10.1109/32.295892
  5. Derek L. Eager, Edward D. Lazowska, and John Zarhorjan. Adaptive load sharing in homogeneous distributed systems, IEEE Transactions on Software Engineering, Vol. 12(5): 662-675, May 1986 https://doi.org/10.1109/TSE.1986.6312961
  6. V. S. Sunderam. PVM: A framework for parallel distributed computing. Concurrency: Practice and Experience, Vol. 2(4): 315-339, December 1990 https://doi.org/10.1002/cpe.4330020404
  7. T.-H. Kim and J. M. Purtilo. Configuration-level Optimization of RPC-based Distributed programs. In Proceedings of the 15th International Conference on Distributed Computing Systems, May 1995 https://doi.org/10.1109/ICDCS.1995.500033
  8. B. Martin, C. Bergan, and B. Russ. PARPC: A system for parallel remote procedure calls. In Proceedings of the International Conferences on Parallel Processing, 1987
  9. P. Tang and P. C. Yew. Processor self-scheduling for multiple nested parallel loops. In Proceedings of International Conference on Parallel Processing, August 1986
  10. T. H. Tzen and L. M. Ni. Dynamic loop scheduling for shared-memory multiprocessors. In Proceedings of International Conference on Parallel Processing, August 1991
  11. S. F. Hummel, E. Schonberg, and L. E. Flynn, Factoring: A method for scheduling parallel loops, Communications of the ACM, Vol. 3(8), August, 1992 https://doi.org/10.1145/135226.135232
  12. M. Cierniak, W. Li and M. J. Zaki. Loop scheduling for heterogeneity, In Proceedings of the 4th International Symposium on High Performance Distributed Computings, August 1995 https://doi.org/10.1109/HPDC.1995.518697
  13. C. P. Kruskal and A. Weiss. Allocating independent subtasks on parallel processors, IEEE Transactions on Software Engineering, Vol. 11(10): 1001-1016, October 1985 https://doi.org/10.1109/TSE.1985.231547
  14. C. D. Polychronopoulos and D. J. Kuck. Guided self-scheduling: A practical scheduling scheme for parallel supercomputers, IEEE Transactions on Computer, Vol. C36-(12): 1425-1439, December 1987 https://doi.org/10.1109/TC.1987.5009495