Journal of Sasang Constitutional Medicine (사상체질의학회지)

The Society Of Sasang Constitutional Medicine (사상체질의학회)
권호 표지
DOI QR코드

DOI QR Code

Sasang Constitution Classification using Convolutional Neural Network on Facial Images

콘볼루션 신경망 기반의 안면영상을 이용한 사상체질 분류

  • Ahn, Ilkoo (KM Data Division, Korea Institute of Oriental Medicine) ;
  • Kim, Sang-Hyuk (KM Data Division, Korea Institute of Oriental Medicine) ;
  • Jeong, Kyoungsik (KM Data Division, Korea Institute of Oriental Medicine) ;
  • Kim, Hoseok (KM Data Division, Korea Institute of Oriental Medicine) ;
  • Lee, Siwoo (KM Data Division, Korea Institute of Oriental Medicine)
  • 안일구 (한국한의학연구원 한의약데이터부) ;
  • 김상혁 (한국한의학연구원 한의약데이터부) ;
  • 정경식 (한국한의학연구원 한의약데이터부) ;
  • 김호석 (한국한의학연구원 한의약데이터부) ;
  • 이시우 (한국한의학연구원 한의약데이터부)
  • Received : 2022.06.28
  • Accepted : 2022.08.10
  • Published : 2022.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Objectives Sasang constitutional medicine is a traditional Korean medicine that classifies humans into four constitutions in consideration of individual differences in physical, psychological, and physiological characteristics. In this paper, we proposed a method to classify Taeeum person (TE) and Non-Taeeum person (NTE), Soeum person (SE) and Non-Soeum person (NSE), and Soyang person (ST) and Non-Soyang person (NSY) using a convolutional neural network with only facial images. Methods Based on the convolutional neural network VGG16 architecture, transfer learning is carried out on the facial images of 3738 subjects to classify TE and NTE, SE and NSE, and SY and NSY. Data augmentation techniques are used to increase classification performance. Results The classification performance of TE and NTE, SE and NSE, and SY and NSY was 77.24%, 85.17%, and 80.18% by F1 score and 80.02%, 85.96%, and 72.76% by Precision-Recall AUC (Area Under the receiver operating characteristic Curve) respectively. Conclusions It was found that Soeum person had the most heterogeneous facial features as it had the best classification performance compared to the rest of the constitution, followed by Taeeum person and Soyang person. The experimental results showed that there is a possibility to classify constitutions only with facial images. The performance is expected to increase with additional data such as BMI or personality questionnaire.

Keywords

Acknowledgement

본 연구는 2022년도 한국한의학연구원 기관주요사업인 "빅데이터 기반 한의 예방치료 원천기술 개발"(Grant No. KSN2023120)의 지원을 받아 수행된 연구임.

References

  1. Lee J. Longevity and Life Preservation in Oriental Medicine (translated by SH Choi). Kyung Hee Univ. Press, Seoul; 1996. (Korean).
  2. Lee HL, Cho JS. Sasang Constitution Classification System by Morphological Feature Extraction of Facial Images. Journal of The Korea Society of Computer and Information. 2015;20(8):15-21. (Korean). DOI: 10.9708/jksci.2015.20.8.015
  3. Cho DU, Kim BH, Lee SH. Sasang Constitution Classification Using Shape Analysis of Face. In Proceedings of the Korea Information Processing Society Conference 2006 (pp. 423-426). Korea Information Processing Society. (Korean).
  4. Cho DU, Kim BH, Lee SH. Sasang Constitution classification system using face morphologic relation analysis. The KIPS Transactions: PartB. 2007; 14(3):153-62. (Korean). DOI: 10.3745/KIPSTB.2007.14-B.3.153
  5. Lee SH, Kim BH, Ka MK, Cho DU, Kwak JH, Oh SY et al. A proposal of Sasang Constitution classification in middle-aged women using image and voice signals process. Journal of the Korea Academia-Industrial cooperation Society. 2008; 9(5):1210-7. (Korean). DOI: 10.5762/KAIS.2008.9.5.1210
  6. Lee SY, Koh BH, Lee EJ, Lee JH, Hwang MW. Systematic review on researches of sasang constitution diagnosis using facial feature. Journal of Sasang Constitutional Medicine. 2012;24(4): 17-27. (Korean). DOI: 10.7730/JSCM.2012.24.4.17
  7. Am I a Taeumin? or a Soyangin?... Represent the Standard Facial Features in the Sasang Constitution. the Dong-A Daily News. 2012.02.13. Available from: URI: http://news.donga.com/3/all/20120113/43282152/1. (Korean).
  8. Do JH, Jang E, Ku B, Jang JS, Kim H, Kim JY. Development of an integrated Sasang constitution diagnosis method using face, body shape, voice, and questionnaire information. BMC complementary and alternative medicine. 2012 Dec; 12(1):1-9. DOI: 10.1186/1472-6882-12-85
  9. Jung C, Yeo I, Jung H. Classification model of facial acne using deep learning. Journal of the Korea Institute of Information and Communication Engineering. 2019;23(4):381-7. (Korean). DOI:10.6109/jkiice.2019.23.4.381
  10. LeCun Y, Bengio Y, Hinton G. Deep learning. nature. 2015 May;521(7553):436-44. DOI: 10.1038/nature14539
  11. Baek YH, Jin HJ, Kim HS, Jang ES, Lee SW. An Overview on the Construction of Korea Constitutional Multicenter Bank for Sasang Constitutional Medicine. Journal of Sasang Constitutional Medicine. 2012;24(2):47-53. (Korean). DOI:10.7730/JSCM.2012.24.2.047
  12. King DE. Dlib-ml: A machine learning toolkit. The Journal of Machine Learning Research. 2009 Dec 1;10:1755-8.
  13. Gonzalez RC, Woods RE. Digital image processing, prentice hall. Upper Saddle River, NJ. 2008.
  14. D. S. Ma, J. Correll, and B. Wittenbrink, "The Chicago face database: A free stimulus set of faces and norming data.," Behav. Res. Methods, vol. 47, pp. 1122-1135, 2015. https://doi.org/10.3758/s13428-014-0532-5
  15. Weiss K, Khoshgoftaar TM, Wang D. A survey of transfer learning. Journal of Big data. 2016 Dec;3(1):1-40. DOI: 10.1186/s40537-016-0043-6
  16. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014 Sep 4. DOI: 10.48550/arXiv.1409.1556
  17. Yoo HJ, Jun KK. Data Augmentation for Image based Parking Space Classification Deep Model. Journal of KIISE (JOK). 2022 Feb;49(2):126-36. DOI: 10.5626/JOK.2022.49.2.126
  18. Hwang DS. Learning Behavior Analysis of Bayesian Algorithm Under Class Imbalance Problems. Journal of the Institute of Electronics Engineers of Korea CI. 2008;45(6):179-86.
  19. Song HW, Park GC, Park J. Prediction of dairy cow mastitis with multi-sensor data using Multi-Layer Perceptron (MLP). In Proceedings of the Korea Information Processing Society Conference 2020 (pp. 788-791). Korea Information Processing Society. DOI: 10.3745/PKIPS.y2020m11a.788
  20. Pham DD, Do JH, Ku B, Lee HJ, Kim H, Kim JY. Body mass index and facial cues in Sasang typology for young and elderly persons. Evidence-Based Complementary and Alternative Medicine. 2011 Jan 1;2011. DOI: 10.1155/2011/749209
  21. Jang ES, Park KH, Baek YH, Lee SW, Kim SH, Joo JC. A Development of Sasang Constitution Diagnosis Program Based on Characteristics of Body Shape Face Physiological Symptom and Personality. Journal of Sasang Constitutional Medicine. 2012;24(1):21-31. (Korean). DOI: 10.7730/jscm.2012.24.1.021
  22. Seok JH, Yoon JH, Lee JH, Hwang MW, Cho YJ, Kho BH et al. The upgrade of Sasangin classification system by the morphologic study of head and face: facial differences on sex and age. Journal of Sasang Constitutional Medicine. 2007;19(3): 30-50. (Korean).