Journal of Intelligence and Information Systems (지능정보연구)

Korea Intelligent Information System Society (한국지능정보시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of Discriminatory Patterns in Performing Arts Recognized by Large Language Models (LLMs): Focused on ChatGPT

거대언어모델(LLM)이 인식하는 공연예술의 차별 양상 분석: ChatGPT를 중심으로

  • Jiae Choi (Department of Global Culture & Management, Calvin University)
  • 최지애 (칼빈대학교 문화경영학과)
  • Received : 2023.09.16
  • Accepted : 2023.09.23
  • Published : 2023.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, the socio-economic interest in Large Language Models (LLMs) has been growing due to the emergence of ChatGPT. As a type of generative AI, LLMs have reached the level of script creation. In this regard, it is important to address the issue of discrimination (sexism, racism, religious discrimination, ageism, etc.) in the performing arts in general or in specific performing arts works or organizations in a large language model that will be widely used by the general public and professionals. However, there has not yet been a full-scale investigation and discussion on the issue of discrimination in the performing arts in large-scale language models. Therefore, the purpose of this study is to textually analyze the perceptions of discrimination issues in the performing arts from LMMs and to derive implications for the performing arts field and the development of LMMs. First, BBQ (Bias Benchmark for QA) questions and measures for nine discrimination issues were used to measure the sensitivity to discrimination of the giant language models, and the answers derived from the representative giant language models were verified by performing arts experts to see if there were any parts of the giant language models' misperceptions, and then the giant language models' perceptions of the ethics of discriminatory views in the performing arts field were analyzed through the content analysis method. As a result of the analysis, implications for the performing arts field and points to be noted in the development of large-scale linguistic models were derived and discussed.

최근 ChatGPT 등의 등장으로 거대언어모델(이하 LLM: Large Language Model)에 대한 사회경제적 관심이 고조되고 있다. 생성형AI의 일종인 거대언어모델은 대본 창착이 가능한 수준까지 이르고 있다. 이러한 측면에서 일반인과 전문가들이 광범위하게 활용할 거대언어모델에서 공연예술 전반 혹은 특정 공연예술물이나 단체의 차별 이슈(성차별, 인종차별, 종교차별, 연령차별 등)를 어떻게 묘사하는지에 관심을 가지고 해결해 나가야 할 것이다. 그러나 아직 거대언어모델에서 공연예술의 차별 이슈에 대한 본격적인 조사와 논의는 이루어지지 않고 있다. 따라서 본 연구의 목적은 거대언어모델로부터의 공연예술 분야 차별이슈 인식 양상을 텍스트 분석하고 이로부터 공연예술분야가 대응할 시사점과 거대언어모델 개발 시사점을 도출하는 것이다. 먼저 거대언어모델에게 차별에 대한 감수성을 측정하기 위해 9가지 차별 이슈에 대한 BBQ(Bias Benchmark for QA) 질문 및 측정법을 사용했으며, 대표적인 거대언어모델로부터 도출된 답변에 대해서 공연예술 전문가에 의해 거대언어모델이 잘못 인지한 부분이 있는지의 검증을 거친 후에 내용분석법을 통해 공연예술분야의 차별적 관점의 윤리성에 대한 거대언어모델의 인식을 분석하였다. 분석 결과로 공연예술 분야에게 주는 시사점과 거대언어모델 개발 시 주의할 점 등을 도출하고 토의하였다.

Keywords

References

  1. Adams, R. (2021). Can artificial intelligence be decolonized?. Interdisciplinary Science Reviews, 46(1-2), 176-197. https://doi.org/10.1080/03080188.2020.1840225
  2. Belk, R. (2021). Ethical issues in service robotics and artificial intelligence. The Service Industries Journal, 41(13-14), 860-876 https://doi.org/10.1080/02642069.2020.1727892
  3. Bellegarda, J. R. (2004). Statistical language model adaptation: review and perspectives. Speech communication, 42(1), 93-108. https://doi.org/10.1016/j.specom.2003.08.002
  4. Bishop, L. (2023). A computer wrote this paper: What chatgpt means for education, research, and writing. Research, and Writing (January 26, 2023).
  5. Breidbach, C. F., & Maglio, P. (2020). Accountable algorithms? The ethical implications of datadriven business models. Journal of Service Management, 31(2), 163-185.
  6. Chen, T. J. (2023). ChatGPT and other artificial intelligence applications speed up scientific writing. Journal of the Chinese Medical Association, 86(4), 351-353. https://doi.org/10.1097/JCMA.0000000000000900
  7. Dev, S., Li, T., Phillips, J. M., & Srikumar, V. (2020, April). On measuring and mitigating biased inferences of word embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 34, No. 05, pp. 7659-7666).
  8. George, J., Mackinnon, A., Kong, D. C., & Stewart, K. (2006). Development and validation of the Beliefs and Behaviour Questionnaire (BBQ). Patient education and counseling, 64(1-3), 50-60.
  9. Houde, S., Liao, V., Martino, J., Muller, M., Piorkowski, D., Richards, J., ... & Zhang, Y. (2020). Business (mis) use cases of generative ai. arXiv preprint arXiv:2003.07679.
  10. Kadiresan, A., Baweja, Y., & Ogbanufe, O. (2022). Bias in AI-based decision-making. In Bridging Human Intelligence and Artificial Intelligence (pp. 275-285). Cham: Springer International Publishing.
  11. Kitamura, F. C. (2023). ChatGPT is shaping the future of medical writing but still requires human judgment. Radiology, 307(2), e230171.
  12. Leavy, S. (2018, May). Gender bias in artificial intelligence: The need for diversity and gender theory in machine learning. In Proceedings of the 1st international workshop on gender equality in software engineering (pp. 14-16).
  13. Malik, T., Dwivedi, Y., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., ... & Wright, R. (2023). "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642.
  14. Marcus, G., & David, E. (2023, January 10). Large language models like ChatGPT say the darnedest things.
  15. McKinsey & Company, 2023, What is generative AI?
  16. Parikh, R. B., Teeple, S., & Navathe, A. S. (2019). Addressing bias in artificial intelligence in health care. Jama, 322(24), 2377-2378.
  17. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training.
  18. Reed, R. (2021). The theology of GPT-2: Religion and artificial intelligence. Religion Compass, 15(11), e12422.
  19. Straw, I. (2020). The automation of bias in medical Artificial Intelligence (AI): Decoding the past to create a better future. Artificial intelligence in medicine, 110, 101965.
  20. Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022, March). Investigating explainability of generative AI for code through scenario-based design. In 27th International Conference on Intelligent User Interfaces (pp. 212-228).
  21. Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., ... & Wang, W. Y. (2019). Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976.
  22. Wahl, B., Cossy-Gantner, A., Germann, S., & Schwalbe, N. R. (2018). Artificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings?. BMJ global health, 3(4), e000798.
  23. Wirtz, J., & Zeithaml, V. (2018). Cost-effective service excellence. Journal of the Academy of Marketing Science, 46(1), 59-80.
  24. 안지인, 장예원, & 정다샘. (2023). 내재된 편향성과 차별을 완화한 한국어 인공지능 동화생성 모델<프랭클린>. 한국 HCI 학회 학술대회, 1338-1341.
  25. 이윤아, & 윤상오. (2022). 인공지능 알고리즘이 유발하는 차별 방지방안에 관한 연구. 한국거버넌스학회보, 29(2), 175-202.