The KIPS Transactions:PartB (정보처리학회논문지B)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Facial Features and Motion Recovery using multi-modal information and Paraperspective Camera Model

다양한 형식의 얼굴정보와 준원근 카메라 모델해석을 이용한 얼굴 특징점 및 움직임 복원

  • 김상훈 (국립한경대학교 제어계측공학과)
  • Published : 2002.10.01
Download PDF
⟨ Previous Next ⟩

Abstract

Robust extraction of 3D facial features and global motion information from 2D image sequence for the MPEG-4 SNHC face model encoding is described. The facial regions are detected from image sequence using multi-modal fusion technique that combines range, color and motion information. 23 facial features among the MPEG-4 FDP (Face Definition Parameters) are extracted automatically inside the facial region using color transform (GSCD, BWCD) and morphological processing. The extracted facial features are used to recover the 3D shape and global motion of the object using paraperspective camera model and SVD (Singular Value Decomposition) factorization method. A 3D synthetic object is designed and tested to show the performance of proposed algorithm. The recovered 3D motion information is transformed into global motion parameters of FAP (Face Animation Parameters) of the MPEG-4 to synchronize a generic face model with a real face.

본 논문은 MPEG4 SNHC의 얼굴 모델 인코딩을 구현하기 위하여 연속된 2차원 영상으로부터 얼굴영역을 검출하고, 얼굴의 특징데이터들을 추출한 후, 얼굴의 3차원 모양 및 움직임 정보를 복원하는 알고리즘과 결과를 제시한다. 얼굴 영역 검출을 위해서 영상의 거리, 피부색상, 움직임 색상정보등을 융합시킨 멀티모달합성의 방법이 사용되었다. 결정된 얼굴영역에서는 MPEG4의 FDP(Face Definition Parameter) 에서 제시된 특징점 위치중 23개의 주요 얼굴 특징점을 추출하며 추출성능을 향상시키기 위하여 GSCD(Generalized Skin Color Distribution), BWCD(Black and White Color Distribution)등의 움직임색상 변환기법과 형태연산 방법이 제시되었다. 추출된 2차원 얼팔 특징점들로부터 얼굴의 3차원 모양, 움직임 정보를 복원하기 위하여 준원근 카메라 모델을 적용하여 SVD(Singular Value Decomposition)에 의한 인수분해연산을 수행하였다. 본 논문에서 제시된 방법들의 성능을 객관적으로 평가하기 위하여 크기와 위치가 알려진 3차원 물체에 대해 실험을 행하였으며, 복원된 얼굴의 움직임 정보는 MPEG4 FAP(Face Animation Parameter)로 변환된 후, 인터넷상에서 확인이 가능한 가상얼굴모델에 인코딩되어 실제 얼굴파 일치하는 모습을 확인하였다.

Keywords

References

  1. MPEG-4 System Sub-group, 'MPEG-4 System Methodology and Work Plan for Scene Description,' ISO/IEC/JTC1/ SC29/WG11/N1786, Jul, 1997
  2. A. Pentland and B. Horowitz. 'Recovery of Non-rigid Motion and Structure,' IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.13, No.7, pp.730-742, 1991 https://doi.org/10.1109/34.85661
  3. M. J. Black and Y. Yaccob, 'Tracking and Recognizing Rigid and Non-rigid Facial Motion using Local Parametric Model of Image Motion,' Proc. Intl Conf. Computer Vision, pp.374-381, 1995 https://doi.org/10.1109/ICCV.1995.466915
  4. A. Azarbayejani and A. Pentland, 'Recursive Estimation of Motion, Structure and Focal Length,' IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.7, No.6, pp.562-575, Jun., 1995 https://doi.org/10.1109/34.387503
  5. J. Weng, N. Ahuja and T. S. Huang, 'Optimal Motion and Structure Estimation,' IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.15, No.9, Sept., 1993 https://doi.org/10.1109/34.232074
  6. T. S. Huang and O. D. Faugeras, 'Some Properties of the E-matrix in Two-view Motion Estimation,' IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.11, No.12, pp.1310-1312, Dec., 1989 https://doi.org/10.1109/34.41368
  7. C. J. Poelman and T. Kanade, 'A Paraperspective Factorization Method for Shape and Motion Recovery,' Technical Report CMU-CS-93-219, Carnegie Mellon University, 1993
  8. B. 0. Jung, 'A Sequential Algorithm for 3-D Shape and Motion Recovery from Image Sequences,' Thesis for the degree of master, Korea University, Jun., 1997
  9. Jibe Yang and Alex Waybill, 'Tracking Human Faces in Real Time,' Technical Report CMU-CS-95-210, Carnage Melon University, 1995
  10. H. Gharavi and Mike Mills 'Blockmatching Motion Estimation Algorithm-New Results,' IEEE Trans. Circuits and System, No.5, Vol.37, 1990
  11. S. H. Kim, H. G. Kim and K. H. Tchah, 'Object-oriented Face Detection using Colour Transformation and Range Segmentation,' IEE Electronics Letters, 14th, Vol.34, No.10, pp.979-980, May, 1998 https://doi.org/10.1049/el:19980714
  12. D. reisfild, 'Detection and Interest Points using Symmetry,' Proc. Intl Conf. Computer Vision, Vol.E8I-D, pp.62-65, Dec., 1990 https://doi.org/10.1109/ICCV.1990.139494
  13. MPEG-4 SNHC Group, 'Face and Body Definition and Animation Parameter,' ISO/IEC JTC1/SC29/WG11 N2202, March, 1998
  14. R. C. Gonzalez and R. E. Woods, 'Digital Image Processing,' Addison-Wesley, pp.225-238, 1992
  15. S. H. Kim and H. G. Kim. 'Face Detection using Multi-Modal Information,' Proc. Intl Conf. Face and Gesture Recognition, France, March, 2000 https://doi.org/10.1109/AFGR.2000.840606