Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Method of Creating Realistic Content in Audience-participating Performances using Artificial Intelligence Sentiment Analysis Technology

인공지능 감정분석 기술을 이용한 관객 참여형 공연에서의 실감형 콘텐츠 생성 방식에 관한 연구

  • Kim, Jihee (Department of IT Media Engineering, Duksung Women's University) ;
  • Oh, Jinhee (Department of IT Media Engineering, Duksung Women's University) ;
  • Kim, Myeungjin (Department of IT Media Engineering, Duksung Women's University) ;
  • Lim, Yangkyu (IT Media Eng. College of Science and Technology, Duksung Women's University)
  • 김지희 (덕성여자대학교 IT미디어공학과) ;
  • 오진희 (덕성여자대학교 IT미디어공학과) ;
  • 김명진 (덕성여자대학교 IT미디어공학과) ;
  • 임양규 (덕성여자대학교 IT미디어공학전공)
  • Received : 2021.07.19
  • Accepted : 2021.08.31
  • Published : 2021.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, a process of re-creating Jindo Buk Chum, one of the traditional Korean arts, into digital art using various artificial intelligence technologies was proposed. The audience's emotional data, quantified through artificial intelligence language analysis technology, intervenes in various object forms in the projection mapping performance and affects the big story without changing it. If most interactive arts express communication between the performer and the video, this performance becomes a new type of responsive performance that allows the audience to directly communicate with the work, centering on artificial intelligence emotion analysis technology. This starts with 'Chuimsae', a performance that is common only in Korean traditional art, where the audience directly or indirectly intervenes and influences the performance. Based on the emotional information contained in the performer's 'prologue', it is combined with the audience's emotional information and converted into the form of images and particles used in the performance to indirectly participate and change the performance.

본 연구에서는 한국 전통예술 중 하나인 진도 북춤을 다양한 인공지능 기술을 이용하여 디지털 아트로 재창조하는 과정을 제안하였다. 인공지능 언어 분석 기술을 통해 정량화된 관객들의 감정 데이터는 프로젝션 맵핑 공연에서 다양한 오브젝트 형태로 개입하여 큰 스토리에 변화를 주지 않는 선에서 영향을 준다. 대부분의 인터렉티브 아트들이 공연자와 영상 간의 소통을 표현한 것이라면, 본 공연은 인공지능 감정분석 기술을 중심으로 관객이 작품과 직접 소통할 수 있는 새로운 형태의 반응형 공연이 된다. 이는 한국 전통예술에서만 흔히 나타나는 관객이 공연에 직간접적으로 개입하여 영향을 끼치는 퍼포먼스인 '추임새'에서 시작된다. 공연자의 '프롤로그'에 담긴 감정 정보를 기반으로 관객의 감정 정보와 결합하여, 공연에 쓰이는 이미지와 파티클의 형태로 변환함으로서 공연에 관객이 간접적으로 참여하고 변화를 줄 수 있는 형태가 된다.

Keywords

Acknowledgement

본 연구는 2021년도 덕성여자대학교 교내연구비 지원에 의해 이루어졌음.

References

  1. Jindo Buk Chum (Jindo Buk Play), https://www.culture.go.kr/knowledge/encyclopediaView.do?code_value=C&vvm_seq=423&ccm_code=C011&ccm_subcode=C211 (accessed July 01, 2021)
  2. Jindo Buk Chum demonstration by Park Gwan-yong, https://youtu.be/yhMw9qjmHpQ (accessed July 01, 2021)
  3. Body, Movement, Language: AI Sketches with Bill T. Jones, https://youtu.be/RVyh1ewep84 (accessed July 01, 2021)
  4. Tensorflow, https://www.tensorflow.org/ (accessed July 01, 2021)
  5. Body, Movement, Language: AI Sketches with Bill T. Jones, https://experiments.withgoogle.com/billtjonesai (accessed July 01, 2021)
  6. TechiEon, https://ars.electronica.art/outofthebox/en/techieon/(accessed July 01, 2021)
  7. [2020 Corea Impact] TechiEon - Tech見, https://youtu.be/fAgg6gMbmg (accessed July 01, 2021)
  8. INSIDE SOUL 'Badabi' ,https://youtu.be/oyz82gTT8WI (accessed July 01, 2021)
  9. Everybody Dance Now, https://arxiv.org/abs/1808.07371 (accessed July 01, 2021)
  10. Everybody Dance Now, https://youtu.be/PCBTZh41Ris (accessed July 01, 2021)
  11. J. Park, Design and implementation of artificial intelligence (AI) technology driven interactive fusion dance performance, 2021, Retrieved from http://hanyang.dcollection.net/public_resource/pdf/200000486219_20210717064955.pdf (accessed July 01, 2021)
  12. 2020 Seven Square 72 Untact Performance_High Quality Version, https://youtu.be/iZ1D2bY3oaQ (accessed July 01, 2021)
  13. H. Kim, H. Oh, D. Kim, "CNN Architecture Predicting Movie Rating from Audience's Reviews Written in Korean," Korea Information Processing Society, Vol. 9, No.1, pp17-24, 2020.
  14. KoNLPy, https://konlpy.org/ko/v0.4.3/morph/ (accessed July 01, 2021)
  15. J. Hong, Y. Jung, "Establishing the category of emotion verb and classifying emotion verbs," Korean Studies, Vol 45, pp.387-420. 2009..
  16. P. Eckman, "Facial Expressions," Handbook og Cognition and Emotion, WILEY, 1999.
  17. Introduction to natural language processing using deep learning, https://wikidocs.net/book/2155 (accessed July 01, 2021)
  18. Plutchik's Wheel of Emotions: Exploring the Emotion Wheel, https://www.6seconds.org/2020/08/11/plutchik-wheel-emotions/?gclid=CjwKCAjw3MSHBhB3EiwAxcaEu1jTbsOR9iZSjg71DaflgBR8A2ESfbKkUHRo8Npl-k1 (accessed July 01, 2021)