IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

Survey on Deep Learning Methods for Irregular 3D Data Using Geometric Information

불규칙 3차원 데이터를 위한 기하학정보를 이용한 딥러닝 기반 기법 분석

  • Received : 2021.08.20
  • Accepted : 2021.09.17
  • Published : 2021.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

3D data can be categorized into two parts : Euclidean data and non-Euclidean data. In general, 3D data exists in the form of non-Euclidean data. Due to irregularities in non-Euclidean data such as mesh and point cloud, early 3D deep learning studies transformed these data into regular forms of Euclidean data to utilize them. This approach, however, cannot use memory efficiently and causes loses of essential information on objects. Thus, various approaches that can directly apply deep learning architecture to non-Euclidean 3D data have emerged. In this survey, we introduce various deep learning methods for mesh and point cloud data. After analyzing the operating principles of these methods designed for irregular data, we compare the performance of existing methods for shape classification and segmentation tasks.

Keywords

Acknowledgement

This research was supported by Samsung Electronics, the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1C1C1009662), and Basic Science Research Program through the NRF funded by Ministry of Education (No. NRF-2020X1A3A1093880)

References

  1. A. Krizhevsky, I. Sutskever, G.E. Hinton, "Imagenet Classification with Deep Convolutional Neural Networks," In Advances in neural information processing systems, pp. 1097-1105, 2012.
  2. J. Long, E. Shelhamer, T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3440, 2015.
  3. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Y. LeCun, "Overfeat: Integrated Recognition, Localization and Detection Using Convolutional networks," arXiv preprint arXiv:1312.6229, 2013.
  4. J.W.H. Tangelder, R.C. Veltkamp, "A Survey of Content Based 3D Shape Retrieval Methods," Multimedia tools and applications, Vol. 39, No. 3, pp. 441-471, 2008. https://doi.org/10.1007/s11042-007-0181-0
  5. K. Michael, T. Funkhouser, S. Rusinkiewicz, "Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors," Symposium on geometry processing, Vol. 6, pp. 156-164, 2003.
  6. Z.B. Liu, S.H. Bu, K. Zhou, S.M. Gao, J.W. Han, J. Wu, "A Survey on Partial Retrieval of 3D Shapes," Journal of Computer Science and Technology, Vol. 28, No. 5, pp. 836-851, 2013. https://doi.org/10.1007/s11390-013-1382-9
  7. Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, J. Xiao, "3D ShapeNets: A Deep Representation for Volumetric Shapes," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912-1920, 2015.
  8. H. Su, S. Maji, E. Kalogerakis, E. Learned-Miller, "Multi-View Convolutional Neural Networks for 3D Shape Recognition," In Proceedings of the IEEE International Conference on Computer Vision, pp. 945-953, 2015.
  9. R. Klokov, V. Lempitsky, "Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models," In Proceedings of the IEEE International Conference on Computer Vision, pp. 863-872, 2017.
  10. E. Ahmed, A. Saint, A. Shabayek, K. Cherenkova, R. Das, G. Gusev, D. Aouada, "A Survey on Deep Learning Advances on Different 3D Data Representations," arXiv preprint arXiv:1808.01462, 2018.
  11. D. Maturana, S. Scherer, "Voxnet: A 3D Convolutional Neural Network for Real-time Object Recognition," IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 922-928, 2015.
  12. M. Gadelha, R. Wang, S. Maji, "Multiresolution Tree Networks for 3D Point Cloud Processing," In Proceedings of the European Conference on Computer Vision, pp. 103-118, 2018.
  13. Y. Feng, Z. Zhang, X. Zhao, R. Ji, Y. Gao, "GVCNN: Group-view Convolutional Neural Networks for 3D Shape Recognition," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 264-272, 2018.
  14. R. Hanocka, A. Hertz, N. Fish, R. Giryes, S. Fleishman, D. Cohenor, "MeshCNN: A Network with an Edge," ACM Transactions on Graphics, Vol. 38, pp. 1-12, 2019.
  15. Y. Feng, Yi. Feng, H. You, X. Zhao, Y. Gao, "Meshnet: Mesh Neural Network for 3D Shape Representation," Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, No. 1, 2019.
  16. C.R. Qi, H. Su, K. Mo, L.J. Guibas, "Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation," Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
  17. Y. Liu, B. Fan, S. Xiang, C. Pan, "Relation-shape Convolutional Neural Network for Point Cloud Analysis," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
  18. M. Jiang, Y. Wu, T. Zhao, Z. Zhao, "PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation," arXiv preprint arXiv:1807.00652, 2018.
  19. J. Li, B.M. Chen, G.H. Lee, "SO-Net: Self-Organizing Network for Point Cloud Analysis," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9397-9406, 2018.
  20. X. Liu, Z. Han, Y.S. Liu, M. Zwicker, "Point2sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network," Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, No. 1, 2019.
  21. I. Kostrikov, Z. Jiang, D. Panozzo, D. Zorin, J. Bruna, "Surface Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2540 - 2548, 2018.
  22. Y. Xu, T. Fan, M. Xu, L. Zeng, Y. Qiao, "Spidercnn: Deep Learning on Point Sets with Parameterized Convolutional Filters," Proceedings of the European Conference on Computer Vision (ECCV), pp. 87-102, 2018.
  23. H. Hoppe, "View-dependent Refinement of Progressive Meshes," In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 189-198, 1997.
  24. M. Jaderberg, K. Simonyan, A. Zisserman, "Spatial Transformer Networks," Advances in neural information processing systems, Vol. 28, pp. 2017-2025, 2015.
  25. C.R. Qi, L. Yi, H. su, L.J. Guibas, "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space," In NeurIPS, pp. 5099-5108, 2017.
  26. L. Landrieu, M. Simonovsky, "Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4558-4567, 2018.
  27. Y. Qang, Y. SUN, "Dynamic Graph cnn for Learning on Point Clouds," ACM Transactions On Graphics, Vol. 38, No. 5, 2019.
  28. Y. Wang, S. Asafi, O.V. Kaick, H. Zhang, D. CohenOr, B Chen, "Active Co-analysis of a set of Shapes," ACM Transactions on Graphics, Vol. 31, No. 6, 2012.
  29. L. Yi, V.G. Kim, D. Ceylan, I.-C. Shen, M. Yan, H. Su, C. Lu, Q. Huang, A. Sheffer, L. Guibas, "A Scalable Active Framework for Region Annotation in 3D Shape Collections," ACM Transactions on Graphics, Vol. 35, No. 6, 2016.