국방Bt&D사업 자체평가시스템 메타평가

A Study on the Meta Evaluation for Defense R&D Programs

  • Kim, Soon-Yeong (Dept. of Technology Evaluation, Defense Agency for Technology and Quality) ;
  • Ha, Kyu-Soo (The Graduate School of Venture, Hoseo University)
  • 발행 : 2009.09.30

초록

This study is the result of meta evaluation for the self evaluation of defense R&D programs in Korea by using meta evaluating indicators. The overall meta evaluation result of defense R&D programs gained 74.3 points out of 100, so it was evaluated as 'Good'. But it demonstrated that further improvement for overall system of defense R&D programs evaluation is required. And especially, it demonstrated that more alternatives are necessary in order to improve the utilizations and the feedbacks of evaluation results. The evaluation context component gained 80.2 points out of 100, so it was evaluated as 'Very Good'. The evaluation input component gained 73.1 points out of 100, so it was evaluated as 'Good'. The evaluation process component gained 74.8 points out of 100, so it was evaluated as 'Good'. And the evaluation outcome component gained 69.0 points out of 100, so it was evaluated as 'Good'. Basic model of meta evaluation was derived from the literature review and brain storming. And this meta evaluation model was determined by adopting the result of experts who performed evaluations for defense R&D programs in recent years. The reliability of components and items was verified by Cronbach's a coefficient. It was over 0.6 in evaluation components and items. And the reliability of evaluation context was 0.877, that of evaluation input was 0.755, that of evaluation process was 0.755, that of evaluation output was 0.755 respectively. From the analysis, it is attempted to identify possible problems and to find out the ways of improvements related to the self evaluation system of defense R&D programs. The ultimate objective of this study is to manage the programs effectively and improve the reliability and the objectiveness of the defense R&D programs.

키워드

참고문헌

  1. 과학기술부; “국가연구개발사업 등의 성과평가 및 성과관리에 관한 법률”, 법률 제7808호, 2005
  2. 과학기술혁신본부; “2008년도 국가연구개발사업 자체평가매뉴얼”, 2008
  3. 김병철, “정부출연연구기관 평가에 대한 메타평가 모형설계 및 적용”, 박사학위논문, 서울시립대학교 대학원, 2008
  4. 노유진, “우리나라 정부부처에 있어서 정책평가결과의 저활용요인에 관한 연구”, 행정논총, 43(4) : 245-276, 2005
  5. 류영수; “기술영향평가의 메타평가 모형개발 및 적용”, 박사학위논문, 한양대학교 대학원, 2007
  6. 박종수; “정보화 사업 메타평가를 위한 CIPOU 모형개발”, 박사학위논문, 충남대학교 대학원, 2003
  7. 방위사업청; “방위력개선사업관리규정”, 2006
  8. 이학식, 임지훈; “SPSS 14.0 매뉴얼”, 법문사, 2008
  9. 이형우; “국가연구개발사업 평가체계에 관한 소고”, 정책분석평가학회보, 15(2) : 191-219, 2005
  10. 이찬구; “영국의 범부처간 과학기술정책 평가제도-메타평가와 한국에의 시사점”, 한국행정논문집, 15(4) : 809-833, 2003
  11. American Evaluation Association "Guidelines for Evaluator," by Shadish W. R., D. L. Newman, M. A. Scheirer and C.; Wye, San francisco, Jossey-Bass, 1995
  12. Chelimsky, Eleanor; "Program Evaluation: Patterns and Directions," 1-35, Washington DC, The American Society for Public Administration, 1985
  13. Cook T D. and Gruder C. L.; "Metaevaluation Research," Evaluation Quarterly, 2(1) : 5-55, 1978 https://doi.org/10.1177/0193841X7800200101
  14. Hong H. D.; "Meta-evaluation of National Large-Scale R&D Programs : A Comparison of Evaluation Systems of 6 National R&D Programs," University of Manchester, 2000
  15. Joint Committee on Standards for Educational Evaluation; "The Programme Evaluation Standards," Thousand Oaks, Sage Publications, 1994
  16. Larson, Richard C and Leni Berliner; "On Evaluating Valuations," Policy Science, 16(2) : 147-163, 1983 https://doi.org/10.1007/BF00138348
  17. Leviton, Laura C and Edward F. X. Hughes; "Research on the Utilization of Evaluations: A Review and Synthesis," Evaluation Review, 5(4) : 1981 https://doi.org/10.1177/0193841X8100500405
  18. OECD, "Improving Evaluation Practices-Best Practice Guidelines for Evaluation and Background Paper," Paris, 1999
  19. Scriven M.; "Evaluation Thesaurus(4th ed.)," Newbury Park, CA, Sage Publications, 1991
  20. Stevenson, John F., Richard H. Longabaugh and Dwight N. Mcneil; "Metaevaluation in Human Services," The Evaluator and Management, Beverly Hills, CA, Sage Publications, 2000
  21. Stufflebeam L.; "Metaevaluation : Concepts, Standard and Uses," Educational Evaluation Methodology, The JHU Press, 1981
  22. Worthen, Blaine R., James R. Sanders and Jody L.; Fitzpatrisk, "Program Evaluation: Alternative Approaches and Practical Guidelines," Second Edition, White Plains, Longman Publishers, 1997
  23. Yi, Chan Goo, "The National R&D Evaluation System in the UK : Meta-evaluation and Applicability to the Korea Case," Thesis for the degree of PhD, PREST, University of Manchester, 2003