Journal of the Korea Academia-Industrial cooperation Society (한국산학기술학회논문지)

The Korea Academia-Industrial cooperation Society (한국산학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Validation Technique of Simulation Model using Weighted F-measure with Hierarchical X-means (WF-HX) Method

계층적 X-means와 가중 F-measure를 통한 시뮬레이션 모델 검증 기법

  • Yang, Dae-Gil (Dept. of Industrial and Management Engineering, Korea University) ;
  • HwangBo, Hun (Industrial and Systems Engineering Division, Texas A&M University) ;
  • Cheon, Hyun-Jae (The Information Security Institute, Korea University) ;
  • Lee, Hong-Chul (Dept. of Information and Management Engineering, Korea University)
  • 양대길 (고려대학교 산업경영공학과) ;
  • 황보훈 (텍사스 A&M 대학교 산업시스템공학과) ;
  • 천현재 (고려대학교 정보보호연구원) ;
  • 이홍철 (고려대학교 정보경영공학부)
  • Received : 2012.01.03
  • Accepted : 2012.02.10
  • Published : 2012.02.29
Download PDF
⟨ Previous Next ⟩

Abstract

Simulation validation techniques which have been employed in most studies are statistical analysis, which validate a model with mean or variance of throughput and resource utilization as an evaluation object. However, these methods have not been able to ensure the reliability of individual elements of the model well. To overcome the problem, the weighted F-measure method was proposed, but this technique also had some limitations. First, it is difficult to apply the technique to complex system environment with numerous values of interarrival time because it assigns a class to an individual value of interarrival time. In addition, due to unbounded weights, the value of weighted F-measure has no lower bound, so it is difficult to determine its threshold. Therefore, this paper propose weighted F-measure technique with cluster analysis to solve these problems. The classes for the technique are defined by each cluster, which reduces considerable number of classes and enables to apply the technique to various systems. Moreover, we improved the validation technique in the way of assigning minimum bounded weights without any lack of objectivity.

기존 대부분의 연구에서 사용하고 있는 시뮬레이션 검증 기법은 통계적 분석기법으로, 총 처리량이나 자원 이용률의 평균 및 분산을 통해 분석하여 왔다. 그러나 이러한 방식은 모델의 개별적인 요소들에 대한 신뢰성을 보장하기 어려웠다. 이를 해결하기 위해 제시된 방법이 가중 F-measure를 사용한 검증이다. 하지만 가중 F-measure는 Tact time 값 하나에 대해 하나의 클래스를 할당하기 때문에 수많은 Tact time 값들을 갖는 복잡한 시스템에 적용하기 어려운 문제를 가지고 있다. 한편, 가중치의 범위가 정해져 있지 않기 때문에 평가기준(Threshold)의 선정에 있어서 어느 정도의 수준이 만족할만한 수준인지 정하기가 어려웠다. 따라서 본 논문에서는 이러한 문제점을 개선하기 위해 군집분석을 적용한 가중 F-measure를 제시한다. 군집의 클래스화를 통해 클래스의 수를 현저히 줄일 수 있고 다양한 시스템으로의 적용 또한 가능해진다. 또한 객관성을 저하시키지 않는 범위 내에서 최소한의 가중치를 부여하는 방식으로 가중치의 범위를 지정하여 검증 방법을 향상시켰다. 이를 입증하기 위해 국내 'L사'의 LCD공정설비를 대상으로 시뮬레이션 모델링 및 환경을 구축하였고, 그 결과를 통해 타당성을 증명하였다.

Keywords

References

  1. A. Kumar, Y. Sabharwal, and S. Sen, "A simple linear time (1+${\varepsilon}$)-approximation algorithm for k-means clustering in any dimensions", Proc. 45th IEEE Symposium on Foundations of Computer Science, pp.454-462, 2004.
  2. Balci, o., "Validation, verification, and testing techniques throughout the life cycle of a simulation study", Annals of Operations Research, Vol.53, pp.121-173, 1994. https://doi.org/10.1007/BF02136828
  3. David Arthur, Sergei Vassilvitskii, "k-means++: the advantages of careful seeding", Proceeding SODA '07 Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pp.1027-1035, 2007.
  4. Dan Pelleg and Andrew Moore, "X-means : Extending K-means with Efficient Estimation of the number of clusters", proceedings of the 17th International Conference on Machine Learning, pp.727-734, 2000.
  5. Harrison, S. R., "Regression of a model on real-system output: An invalid test of model validity", Agricultural systems, Vol.34, No.3, pp.183-190, 1990. https://doi.org/10.1016/0308-521X(90)90083-3
  6. Hwangbo hun , Cheon hyeon-jae , Lee hong-cheol, "Using the weighted F-measure of Trace-Driven Simulation model verification methods," Journal of Simulation, Vol.18, No.4, pp.185-195, 2009.
  7. Jack P.C. Kleijnen, "Verification and Validation of Simulation Models", European J. of Operational Research, pp.82, 145-162, 1995. https://doi.org/10.1016/0377-2217(94)00016-6
  8. Jack P.C. Kleijnen, Bert Bettonvil, Willem Van Groenendahl, "Validation of Trace -Driven Simulation Models : Regression Analysis Revised", Proceedings of the WSC, pp.352-359, 1994.
  9. Lee young-hae, Gwak seong-geun, Kim suk-han, "Under a distributed environment, research on Web-based simulation", Journal of Korea Society for Simulation, No. 7 No. 2, pp.79-90, 1998.
  10. L. Excoffier and M. Slatkin, "Maximum - Likelihood Estimation of Molecular Haplotype Frequencies in a Diploid Population", Oxford Journals Life Sciences & Medicine Molecular Biologyand Evolution, Volume12, Issue5, pp. 921-927, 1995.
  11. L. Yen, D. Vanvyve, F. Wouters, F. Fouss, M. Verleysen, M. Saerens, "Clustering using a random walk based distance measure", proceedings - European Symposium on Artificial Neural Networks Bruges April, pp.317-324, 2005.
  12. Lee young-hae, Choe gwan-grim, "Intervention Analysis techniques, utilizing research on Validation of simulation models," Journal of Korea Society for Simulation, pp.175, 1997.
  13. Murray-Smith, D. J., "Methods for the external validation of continuous system simulation models", Mathematical and computer modelling of dynamic systems, Vol.4, No.1, pp.5-31, 1998. https://doi.org/10.1080/13873959808837066
  14. S. Lloyd, "Least Squares Quantization in PCM", IEEE Transactions on Information Theory, Vol.28, pp.129-137, 1982. https://doi.org/10.1109/TIT.1982.1056489
  15. S. N. Crozier, D. D. Falconer, S. A. Mahmoud, "Least sum of squared errors (LSSE) channel estimation", Radar and Signal Processing, IEEE Proceedings F, Volume 138, Issue 4, pp.371-378, 1991. https://doi.org/10.1049/ip-f-2.1991.0048
  16. Sargent, R. G., "Verification and Validation of Simulation Models", Proceedings of the 2010 Winter Simulation Conference, pp.166-183, 2010.
  17. Smith, E. P. and Rose, K. A., "Model goodness- of-fit analysis using regression and related techniques", Ecological modelling, Vol.77, No.1, pp.49-64, 1995. https://doi.org/10.1016/0304-3800(93)E0074-D
  18. Wei, X. C. and Li, E. P., "Reflection of transmitting antenna in reverberation chamber and its effect on chi-square validation", Antennas and propagation society international symposium 2006, pp.3573-3576, 2006.
  19. Box G. E. P, Jenkins G. M, and Reinsel G. C., "Time Series Analysis ; Forecasting and Control", pp.463-479, Prentice Hall 3rd, 1994.
  20. Kang seok-bok, "Statistical estimation and hypothesis testing", pp.156-318, Gyeongmunsa, 2002.
  21. Kim dae-hak, "Utilizing spss, analysis of variance", pp.39-65, Gyowoosa, 2004.
  22. Ross, S. M., Simulation 4th Ed., pp.294-306, Elsevier Academic Press, 2006.
  23. Law, A. M. and Kelton, W. D., "Simulation modeling and analysis 3rd Ed.", pp.283-290, McGraw-Hill, 2000.