Investigative Magnetic Resonance Imaging

Korean Society of Magnetic Resonance in Medicine (대한자기공명의과학회)
권호 표지
DOI QR코드

DOI QR Code

Unsupervised Deformable Image Registration Using Polyphase UNet for 3D Brain MRI Volumes

  • Martin, Antoinette D. (Korea Advanced Institute of Science and Technology (KAIST)) ;
  • Kim, Boah (Korea Advanced Institute of Science and Technology (KAIST)) ;
  • Ye, Jong Chul (Korea Advanced Institute of Science and Technology (KAIST))
  • Received : 2020.07.06
  • Accepted : 2020.11.17
  • Published : 2020.12.31
⟨ Previous Next ⟩

Abstract

Purpose: Image registration is a fundamental task in various medical imaging studies and clinical image analyses, such as comparison of patient data with anatomical structures. In order to solve the problems of conventional image registration approaches, such as long computational time, recent deep-learning supervised and unsupervised methods have been extensively studied because of their excellent performance and fast computational time. In this study, we propose a deep-learning-based network for deformable medical image registration using unsupervised learning. Materials and Methods: In this paper, we solve the image-registration optimization problem by modelling a function using a convolutional neural network with polyphase decomposition to learn the spatial transformable parameters based on the input images and to generate the registration field. A spatial transformer is used to reconstruct the output warped image while imposing smoothness constraints on the registration field. With polyphase decomposition, our proposed method learns more features based on the input image pairs without the need for any ground-truth registration field. Results: Experimental results using 3D T1 brain MRI volume scans and compared with state-of-the-art image-registration methods demonstrated that our method provides better 3D-image registration. Conclusion: Our proposed method uses less computational time in registering unseen pairs of input images during inference and can be applied for other unimodal image registration tasks, and the hyper-parameters can be adjusted for the specific task.

Keywords

References

  1. Boveiri HR, Khayami R, Javidan R, MehdiZadeh AR. Medical image registration using deep neural networks: a comprehensive review. ArXiv 2002.03401;1-45
  2. Haskins G, Kruger U, Yan P. Deep learning in medical image registration: a survey. Mach Vision Appl 2020;31:8 https://doi.org/10.1007/s00138-020-01060-x
  3. Wu G, Kim M, Wang Q, Gao Y, Liao S, Shen D. Unsupervised deep feature learning for deformable registration of MR brain images. Med Image Comput Comput Assist Interv 2013;16:649-656
  4. Ghosal S, Ray N. Deep deformable registration: enhancing accuracy by fully convolutional neural net. Pattern Recognit Lett 2017;94:81-86 https://doi.org/10.1016/j.patrec.2017.05.022
  5. Blendowski M, Heinrich MP. 3D-CNNs for deep binary descriptor learning in medical volume data. Informatik Aktuell 2018:23-28
  6. Liu X, Jiang D, Wang M, Song Z. Image synthesis-based multi-modal image registration framework by using deep fully convolutional networks. Med Biol Eng Comput 2019;57:1037-1048 https://doi.org/10.1007/s11517-018-1924-y
  7. Cheng X, Wang B, Liu Y, Yuan Y, Shu Y, Chen B. Comparable electrode impedance and speech perception at 12 months after cochlear implantation using round window versus cochleostomy: an analysis of 40 patients. ORL J Otorhinolaryngol Relat Spec 2018;80:248-258 https://doi.org/10.1159/000490764
  8. Yang X, Kwitt R, Niethammer M. Fast predictive image registration. Lect Notes Comput Sc 2016:48-57
  9. Sokooti H, de Vos B, Berendsen F, Lelieveldt BPF, Isgum I, Staring M. Nonrigid image registration using multi-scale 3D convolutional neural networks. Lect Notes Comput Sc 2017:232-239
  10. Eppenhof KAJ, Lagarge MW, Moeskops P, Veta M, Pluim JPW. Deformable image registration using convolutional neural networks. Med Imaging 2018: Image Processing 2018:105740S
  11. Cao X, Yang J, Zhang J, et al. Deformable image registration based on similarity-steered CNN regression. Med Image Comput Comput Assist Interv 2017;10433:300-308
  12. Sun L, Zhang S. Deformable MRI-ultrasound registration using 3D convolutional neural network. Lect Notes Comput Sc 2018:152-158
  13. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV. VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging 2019 [Online ahead of print]
  14. Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K. Spatial transformer networks. Advances in Neural Information Processing Systems, 2015:2017-2025
  15. Li H, Fan Y. Non-rigid image registration using selfsupervised fully convolutional networks without training data. IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018:1075-1078
  16. de Vos BD, Berendsen FF, Viergever MA, Staring M, Isgum I. End-to-end unsupervised deformable image registration with a convolutional neural network. Lect Notes Comput Sc, 2017:204-212
  17. Dalca AV, Balakrishnan G, Guttag J, Sabuncu MR. Unsupervised learning for fast probabilistic diffeomorphic registration. Lect Notes Comput Sc, 2018:729-738
  18. Yoo I, Hildebrand DGC, Tobin WF, Lee WCA, Jeong WK. ssEMnet: serial-section electron microscopy image registration using a spatial transformer network with learned features. Lect Notes Comput Sc 2017:249-257
  19. Shu C, Chen X, Xie Q, Han H. An unsupervised network for fast microscopic image registration. Medical Imaging 2018: Digital Pathology 2018;10581:105811D
  20. Kearney V, Haaf S, Sudhyadhom A, Valdes G, Solberg TD. An unsupervised convolutional neural network-based algorithm for deformable image registration. Phys Med Biol 2018;63:185017 https://doi.org/10.1088/0031-9155/63/18/185017
  21. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV. An unsupervised learning model for deformable medical image registration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018:9252-9260
  22. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. MICCAI 2015: Medical Image Computing and Computer-Assisted Intervention, 2015:234-241
  23. Kim B, Ye JC. Cycle-consistent adversarial network with polyphase U-Nets for liver lesion segmentation. 1st Conf Med Imaging with Deep Learn (MIDL 2018), 2018:1-3
  24. Ye JC, Han Y, Cha E. Deep convolutional framelets: a general deep learning framework for inverse problems. SIAM J Imaging Sci 2018;11:991-1048 https://doi.org/10.1137/17M1141771
  25. Kim B, Ye JC. Mumford-shah loss functional for image segmentation with deep learning. IEEE T Image Process 2019;29:1856-1866 https://doi.org/10.1109/TIP.2019.2941265
  26. LaMontagne PJ, Benzinger TLS, Morris JC, et al. OASIS-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and Alzheimer disease. Alzheimer's Dement 2018;14:P1097
  27. Fischl B. FreeSurfer. Neuroimage 2012;62:774-781 https://doi.org/10.1016/j.neuroimage.2012.01.021
  28. Avants BB, Epstein CL, Grossman M, Gee JC. Symmetric diffeomorphic image registration with crosscorrelation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 2008;12:26-41 https://doi.org/10.1016/j.media.2007.06.004