Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-sensor Image Registration Using Normalized Mutual Information and Gradient Orientation

정규 상호정보와 기울기 방향 정보를 이용한 다중센서 영상 정합 알고리즘

  • Ju, Jae-Yong (Dept. of Electrical Engineering, Korea University) ;
  • Kim, Min-Jae (Dept. of Electrical Engineering, Korea University) ;
  • Ku, Bon-Hwa (Dept. of Visual Information Processing, Korea University) ;
  • Ko, Han-Seok (Dept. of Electrical Engineering, Korea University)
  • 주재용 (고려대학교 전기전자전파공학과) ;
  • 김민재 (고려대학교 전기전자전파공학과) ;
  • 구본화 (고려대학교 영상정보처리학과) ;
  • 고한석 (고려대학교 전기전자전파공학과)
  • Received : 2012.02.01
  • Accepted : 2012.04.18
  • Published : 2012.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Image registration is a process to establish the spatial correspondence between the images of same scene, which are acquired at different view points, at different times, or by different sensors. In this paper, we propose an effective registration method for images acquired by multi-sensors, such as EO (electro-optic) and IR (infrared) sensors. Image registration is achieved by extracting features and finding the correspondence between features in each input images. In the recent research, the multi-sensor image registration method that finds corresponding features by exploiting NMI (Normalized Mutual Information) was proposed. Conventional NMI-based image registration methods assume that the statistical correlation between two images should be global, however images from EO and IR sensors often cannot satisfy this assumption. Therefore the registration performance of conventional method may not be sufficient for some practical applications because of the low accuracy of corresponding feature points. The proposed method improves the accuracy of corresponding feature points by combining the gradient orientation as spatial information along with NMI attributes and provides more accurate and robust registration performance. Representative experimental results prove the effectiveness of the proposed method.

영상정합은 동일한 장면에 대해서 서로 다른 시점, 서로 다른 시간 혹은 서로 다른 특성의 센서로부터 얻은 영상들의 위치 관계를 대응 시켜주는 기법이다. 본 논문에서는 가시광선 영상 및 적외선 영상과 같은 다중센서 영상을 정합하기 위한 방법을 제안한다. 영상정합은 두 영상에서 특징점을 추출하고, 특징점 간의 대응 관계를 구함으로써 이루어진다. 기존의 다중센서 영상 정합을 위한 방법으로 정규상호정보를 이용하여 대응 특징점을 선별하는 방법이 제안되었다. 정규상호정보 기반의 영상정합 기법은 두 영상의 통계적 상관성이 전역적이어야 한다는 가정을 전제한다. 그러나 가시광선 영상과 적외선 영상에서는 이를 보장하지 못하는 경우가 많아 대응 특징점의 정확도가 저하되기 때문에 기존의 방법은 안정적인 정합 성능을 기대하기 힘들다. 본 논문에서는 영상의 공간정보로서 기울기 방향정보를 정규상호정보와 결합함으로써, 대응 특징점의 정확도를 향상시켰으며 이를 통해 정확성 및 안정적인 영상 정합 결과를 도모하였다. 다양한 실험 결과를 통해 제안하는 방법의 효용성을 증명하였다.

Keywords

References

  1. M. J. Kim, H. S. Ko, et al, "Robust Video Super Resolution Algorithm Using Measurement Validation Method and Scene Change Detection," EURASIP Journal on Advances in Signal Processing, Vol. 2011, No. 103, Nov. 2011.
  2. B. Zitova and J. Flusser, "Image Registration Methods : A Survey," Image and Vision Computing, pp. 977-1000, June. 2003.
  3. J. P. W. Pluim, J. B. A., Maintz, and M. A., Viergever, "Mutual-Information-Based Registration of Medical Images : A Survey," IEEE Transactions on Medical Imaging, Vol. 22, No. 8, pp. 986-1004, Aug. 2003. https://doi.org/10.1109/TMI.2003.815867
  4. F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, "Multimodality Image Registration by Maximization of Mutual Information," IEEE Transactions on Medical Imaging, Vol. 16, No. 2, pp. 187-198, Apr. 1997. https://doi.org/10.1109/42.563664
  5. H. Xie, L. E. Pierce, and F. T. Ulaby, "Mutual Information based Registration of SAR images," in Proc. of IEEE International Geoscience and Remote Sensing Symposium, Vol. 6, pp.4028-4031, July. 2003.
  6. Zeger F. Knops, J. B. A. Maintz, M. A. Viergever and J. P. W. Pluim, "Registration using Segment Intensity Remapping and Mutual Information," International Conference on Medical Image Computing Computer-Assisted Intervention, pp. 805-812, Sep. 2004.
  7. Y. S. Kim, J. H. Lee, and J. B. Ra, "Multi-sensor Image Registration based on Intensity and Edge Orientation Information," Pattern Recognition, Vol. 41, pp. 3356-3365, Apr. 2008. https://doi.org/10.1016/j.patcog.2008.04.017
  8. J. H. Lee, Y. S. Kim, et al, "Robust CCD and IR Image Registration Using Gradient-Based Statistical Information," IEEE Signal Processing Letters, Vol. 17, No. 4, Apr. 2010.
  9. M. Irani and P. Anandan, "Robust Multi-Sensor Image Alignment", Proceedings of IEEE International Conference on Computer Vision, pp. 959-966, Jan. 1998.
  10. W. C. Choi, S. N. Choi, et al, "Automatic Image Registration Based on Extraction of Corresponding-Points for Multi-Sensor Image Fusion," Journal of the Korea Institute of Military Science and Technology, Vol. 12, No. 4, pp. 524-531, Aug. 2009.
  11. Juan Kang, et al, "Image Registration Based on Harris corner and Mutual Information," International Conference on Electronic & Mechanical Engineering and Information Technology, Aug. 2011.
  12. C. Harris and M. J. Stephens, "A Combined Corner And Edge Detector," Alvey Vision Conference, pp.147-151, 1988.
  13. Fischler M A, Bolles R C, "Random Sample Consensus: A Paradigm For Model Fitting With Applications To Image Analysis And Automated Cartography," Communications of the ACM, pp. 381-395, June. 1981.
  14. O. Chum, J. Matas, and J. Kittler, "Locally Optimized RANSAC," Proc. Ann. Symp. German Assoc. for Pattern Recognition (DAGM '03), Sep. 2003.
  15. S. M. Smith, J. M. Brady, "SUSAN-A New Approach to Low Level Image Processing," International Journal of Computer Vision 23, Vo. 23, No. 1, pp. 45-78, Nov. 1997. https://doi.org/10.1023/A:1007963824710
  16. D. G. Lowe, "Distinctive Image Features from Scale- Invariant Keypoints," International Journal of Computer Vision, Vol. 60, No. 2, Oct. 2004.
  17. Rafael C. Gonzalez, Richard E. Woods, Digital Image Processing-3/E, PEARSON, 2009.
  18. H. Farid, and E. P. Simoncelli, "Differentiation of Discrete Multidimensional Signals," IEEE Transactions on Image Processing, Vol. 13, No. 4, Apr. 2004.
  19. J. P. W. Pluim, J. B. A. Maintz, M. A. Viergever, "Image Registration by Maximization of Combined Mutual Information and Gradient Information," IEEE Transactions on Medical Imaging Vol. 19, pp. 809-814, Feb. 2000. https://doi.org/10.1109/42.876307
  20. C.R. Maurer Jr., R.J. Maciunas, J.M. Fitzpatrick, "Registration of Head CT Images to Physical Space using a Weighted Combination of Points and Surfaces," IEEE Transactions on Medical Imaging, Vol. 17, pp. 753-761, Oct. 1998 https://doi.org/10.1109/42.736031
  21. J. Canny, "A Computational Approach to Edge Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 8, pp. 679-698, Nov. 1986 https://doi.org/10.1109/TPAMI.1986.4767851