Journal of Korean Library and Information Science Society (한국도서관정보학회지)

Korean Library and Information Science Society (한국도서관정보학회)
권호 표지
DOI QR코드

DOI QR Code

Quality Factor: A new Bibliometric Measure for Assessing the Quality of Faculty Research Performance

Quality Factor: 교수연구업적평가를 위한 새로운 계량 지표

  • 최은주 (경북대학교 일반대학원 문헌정보학과) ;
  • 양기덕 (경북대학교 사회과학대학 문헌정보학과) ;
  • 이혜경 (경북대학교 일반대학원 문헌정보학과)
  • Received : 2016.05.20
  • Accepted : 2016.06.17
  • Published : 2016.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper introduces a new bibliometric measure called Quality Factor, which assesses multiple facets of faculty research performance. The computation of Quality factor is based on a combination of publication count, citation count, h-index, and Impact Factor. In order to analyze the relationship between Quality Factor and other bibliometric measures (publication count, citation count, h-index, g-index, Impact Factor), the study collected publication data of 189 Korean Library and Information Science professors from 2001 to 2014 to produce the rankings of the faculty by each bibliometric measure and computed Spearman's rank correlations between the rankings. The overall results showed Quality Factor to be correlated to citation-driven measures (citation count, h-index, g-index), but the scatterplot as well as rank-interval analysis showed Quality Factor to be distinctive and more discriminating than other measures.

본 연구는 교수의 연구업적을 보다 다면적으로 측정하기 위한 합리적 방법을 고안하기 위하여 새로운 논문계량지표인 Quality Factor를 개발하였다. Quality Factor는 총 논문수, 피인용수, h-index, Impact Factor를 이용하여 수치를 도출한다. 새로 개발한 지표인 Quality Factor와 다양한 지표(논문 수, Impact Factor, 피인용수, h-index, g-index)간의 관계를 분석하기 위하여, 본 연구는 국내 문헌정보학과 교수 189명의 14년간(2001-2014)의 학술논문데이터를 바탕으로 그들의 지표별(논문 수, Impact Factor, 피인용수) 순위를 산출하고 지표들 간에 스피어만 순위상관관계를 도출하였다. 그 결과, Quality Factor는 전체적으로 피인용 수의 영향을 주로 받는 지표(피인용수, h-index, g-index)와 유사한 상관관계를 보였으나, 산점도에서 Quality Factor가 타 지표에 비하여 분산된 분포를 나타내면서 개별식별성이 높은 것으로 드러났으며 군집 분할 분석에서도 타 지표들과 상이한 상관관계를 보여주었다.

Keywords

References

  1. 강대신, 문성빈 (2009). 연구성과의 질적 평가를 위한 계량정보학적 분석에 관한 연구. 정보관리학회지, 26(3): 377-394.(Kang, Dae-shin and Moon, Sung-Been. 2009. "A Study on Informetric Analysis for Measuring the Qualitative Research Performance." Journal of the Korean Society for Information Management, 26(3): 377-394.)
  2. 김수경, 이숙정.(2012). 대학교수의 연구역량 평가 준거 및 핵심 지표 개발. 교육행정학연구, 30(2): 207-226.(Kim, Soo-Kyung and Lee, Sook-Jeong. 2012. "Development of evaluation criteria and key indicators of research competence in university professors." The Journal of Educational Administration, 30(2): 207-226.)
  3. 김왕준, 윤홍주, 나민주. (2012). 국립대학 교수업적평가 관련규정 비교분석. 한국교원교육(Kim, Wang-Jun and Yoon, Hong-Joo and Rah, Min-Joo. 2012. "A Comparative Analysis of Faculty Evaluation Systems of National Universities in Korea." The Journal of Korean Teacher Education, 29(1): 143-165.)
  4. 백종윤. (2009). 동료평가의 예측가능성 연구: 학술지 게재논문평가를 중심으로. 한국공공관리학보, 23(4): 383-399.(Baek, J. 2009. "Does Peer Review for Journals Predict Performance of Research?" Korean Public Management Review, 23(4): 383-399.)
  5. 이종욱, & 양기덕. (2011). 교수연구업적 평가법의 계량적 분석: 국내 문헌정보학과 교수연구업적을 중심으로. 정보관리학회지, 28(4): 119-140.(Lee, Jong-Wook and Yang, K. 2011. "A Bibliometric Analysis of Faculty Research Performance Assessment Methods." Journal of the Korean Society for Information Management, 28(4): 119-140.)
  6. 이재윤. (2006). 연구성과 측정을 위한 h-지수의 개량에 관한 연구. 정보관리학회지, 23(3): 167-186.(Lee, Jae-Yun. 2006. "Some Improvements on H-Index: Measuring Research Outputs by Citations." Journal of the Korean Society for Information Management, 23(3): 167-186.) https://doi.org/10.3743/KOSIM.2006.23.3.167
  7. 이혜경, & 양기덕. (2015). 국내대학의 연구업적평가기준 비교 분석- 학술논문업적을 중심으로, 한국정보관리학회 학술대회 논문집, 22: 17-22.(Lee, H and Yang K. 2015 "Comparative analysis of Korean universities' research performance evaluation standards : based on journal publications." Proceedings of the 22th Korean Society for Information Management 2015, 17-22)
  8. 전희국, 임동혁, & 김형주. (2012). 시맨틱 링크를 사용한 페이지순위 알고리즘 개선. 정보과학회논문지: 컴퓨팅의 실제 및 레터, 18(12): 843-847.(Jun, Hee-Gook and Im, Dong-Hyuk and Kim, Hyoung-Joo. 2012. "Improvement of PageRank Algorithm using Sementic Web." KTCP, 18(12): 843-847.)
  9. Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2009). "Differences in impact factor across fields and over time." Journal of the American Society for Information Science and Technology, 60(1): 27-34. https://doi.org/10.1002/asi.20936
  10. Bador, P., & Lafouge, T. (2009). "Comparative analysis between impact factor and h-index for pharmacology and psychiatry journals." Scientometrics, 84(1): 65-79. https://doi.org/10.1007/s11192-009-0058-2
  11. Bergstrom, C. T., West, J. D., & Wiseman, M. A. (2008). "The $Eigenfactor^{TM}$ metrics." The Journal of Neuroscience, 28(45): 11433-11434. https://doi.org/10.1523/JNEUROSCI.0003-08.2008
  12. Bornmann, L., Mutz, R., & Daniel, H. D. (2008). "Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine." Journal of the American Society for Information Science and Technology, 59(5): 830-837. https://doi.org/10.1002/asi.20806
  13. COSTAS, Rodrigo; BORDONS, Maria. (2007). "The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level." Journal of Informetrics, 1.3: 193-203. https://doi.org/10.1016/j.joi.2007.02.001
  14. Costas, R., & Bordons, M. (2008). "Is g-index better than h-index? An exploratory study at the individual level." Scientometrics, 77(2): 267-288. https://doi.org/10.1007/s11192-007-1997-0
  15. Cronin, B., & Meho, L. (2006). "Using the h‐index to rank influential information scientistss." Journal of the American Society for Information Science and Technology, 57(9): 1275-1278. https://doi.org/10.1002/asi.20354
  16. Egghe, L. (2006). "An improvement of the h-index: the g-index." ISSI newsletter, 2(1): 8-9.
  17. Garfield, E. (1972). Citation analysis as a tool in journal evaluation. American Association for the Advancement of Science.
  18. GARFIELD, Eugene.(1983). HOW TO USE CITATION ANALYSIS FOR FACULTY EVALUATIONS, AND WHEN IS IT RELEVANT. 1. Current Contents, 1983, 44: 5-13.
  19. GARFIELD, Eugene.(1983). HOW TO USE CITATION ANALYSIS FOR FACULTY EVALUATIONS, AND WHEN IS IT RELEVANT. 2. Current Contents, 1983, 45: 5-14.
  20. GLANZEL, Wolfgang; MOED, Henk F.(2002). "Journal impact measures in bibliometric research." Scientometrics, 53(2): 171-193. https://doi.org/10.1023/A:1014848323806
  21. Gonzalez-Pereiraa, B., Guerrero-Boteb, V. P., & Moya-Anegonc, F. (2009). The SJR indicator: A new indicator of journals' scientific prestige. arXiv preprint arXiv:0912.4141.
  22. HIRSCH, Jorge E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National academy of Sciences of the United States of America. 2005. 102(46): 16569-16572. https://doi.org/10.1073/pnas.0507655102
  23. Hodge, D. R., & Lacasse, J. R. (2010). Evaluating journal quality: Is the H-index a better measure than impact factors?. Research on Social Work Practice.
  24. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). "The R-and AR-indices: Complementing the h-index." Chinese science bulletin, 52(6): 855-863. https://doi.org/10.1007/s11434-007-0145-9
  25. Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). "Bias in peer review." Journal of the American Society for Information Science and Technology, 64(1): 2-17. https://doi.org/10.1002/asi.22784
  26. Li, J., Sanderson, M., Willett, P., Norris, M., & Oppenheim, C. (2010). "Ranking of library and information science researchers: Comparison of data sources for correlating citation data, and expert judgments." Journal of Informetrics, 4(4): 554-563. https://doi.org/10.1016/j.joi.2010.06.005
  27. Luwel, M., Noyons, E., & Moed, H. F. (1999). "Bibliometric assessment of research performance in Flanders: policy background and implications." R&D Management, 29(2): 133-142. https://doi.org/10.1111/1467-9310.00124
  28. Meho, L. I., & Yang, K. (2007). "Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar." Journal of the americn society for information science and technology, 58(13): 2105-2125. https://doi.org/10.1002/asi.20677
  29. Merton, R. K. (1988). "The Matthew effect in science, II: Cumulative advantage and the symbolism of intellectual property." Isis, 606-623.
  30. Moed, H. F. (2010). "Measuring contextual citation impact of scientific journals." Journal of Informetrics, 4(3): 265-277. https://doi.org/10.1016/j.joi.2010.01.002
  31. Moed, H. F., Colledge, L., Reedijk, J., Moya-Anegon, F., Guerrero-Bote, V., Plume, A., & Amin, M. (2012). "Citation-based metrics are appropriate tools in journal assessment provided that they are accurate and used in an informed way." Scientometrics, 92(2): 367-376. https://doi.org/10.1007/s11192-012-0679-8
  32. Van Leeuwen, T. N., Visser, M. S., Moed, H. F., Nederhof, T. J., & Van Raan, A. F. (2003). "The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence." Scientometrics, 57(2): 257-280. https://doi.org/10.1023/A:1024141819302
  33. Van Raan, A. F. (1996). "Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises." Scientometrics, 36(3): 397-420. https://doi.org/10.1007/BF02129602
  34. Van Raan, A. F. (2006). "Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups." scientometrics, 67(3): 491-502. https://doi.org/10.1556/Scient.67.2006.3.10
  35. Waltman, L., Yan, E., & van Eck, N. J. (2011). "A recursive field-normalized bibliometric performance indicator: An application to the field of library and information science." Scientometrics, 89(1), 301-314. https://doi.org/10.1007/s11192-011-0449-z
  36. Wouters, P. F. (1999). The citation culture. Doctoral Dissertation. University of Amsterdam.
  37. Yang, K., & Lee, J. (2013). "Bibliometric Approach to Research Assessment: Publication Count, Citation Count, & Author Rank." Journal of Information Science Theory and Practice, 1(1): 27-41. https://doi.org/10.1633/JISTaP.2013.1.1.2