IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

Development of AI Service with Surgical Tools Segmentation and Action Recognition

수술 도구의 세분화와 행동 인식 기능이 탑재된 AI 서비스 개발

  • Received : 2021.02.15
  • Accepted : 2021.03.29
  • Published : 2021.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose an artificial intelligence (AI) service that plays a supportive role in robot assisted-surgery using deep learning algorithm that have recently been spotlighted in several fields. The proposed AI service is equipped with the ability to segment surgical tools and the ability to recognize the behavior of surgical tools. In addition, such AI service is opened using public web page to make them easier for surgeons to use. Models mounted on AI service are segmentation deep learning model and action recognition deep learning model. The segmentation deep learning model showed a final mIoU performance of 0.867 for seven surgical tools, and the action recognition deep learning model shows an accuracy of 86.96% for the opening and closing actions of all surgical tools.

Keywords

Acknowledgement

본 논문은 정부 (과학기술정보통신부)의 재원으로 한국연구재단의 지원 (No. 2020R1C1C1007423)과 서울대학교병원의 전립선암 수술 영상 데이터를 지원 (No. 2008-169-115) 받아 수행된 연구임.

References

  1. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg and Li Fei-Fei "Imagenet large scale visual recognition challenge." International journal of computer vision 115.3 (2015): 211-252. https://doi.org/10.1007/s11263-015-0816-y
  2. Krizhevsky, Alex, Ilya Sutskever, Geoffrey E. Hinton. "ImageNet classification with deep convolutional neural networks." Communications of the ACM 60.6 (2017): 84-90. https://doi.org/10.1145/3065386
  3. Simonyan, Karen, Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014).
  4. K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
  5. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich. "Going deeper with convolutions." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
  6. Gao Huang, Zhuang Liu, Laurens van der Maaten, Kilian Q. Weinberger. "Densely connected convolutional networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  7. Tan, Mingxing, Quoc Le. "Efficientnet: Rethinking model scaling for convolutional neural networks." International Conference on Machine Learning. PMLR, 2019.
  8. Long, Jonathan, Evan Shelhamer, Trevor Darrell. "Fully convolutional networks for semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
  9. Ronneberger, Olaf, Philipp Fischer, Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015.
  10. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L. Yuille. "Semantic image segmentation with deep convolutional nets and fully connected crfs." arXiv preprint arXiv:1412.7062 (2014).
  11. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L. Yuille. "Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs." IEEE transactions on pattern analysis and machine intelligence 40.4 (2017): 834-848. https://doi.org/10.1109/TPAMI.2017.2699184
  12. Liang-Chieh Chen, George Papandreou, Florian Schroff, Hartwig Adam. "Rethinking atrous convolution for semantic image segmentation." arXiv preprint arXiv:1706.05587 (2017).
  13. Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, Hartwig Adam. "Encoder-decoder with atrous separable convolution for semantic image segmentation." Proceedings of the European conference on computer vision (ECCV). 2018.
  14. Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, Kaiming He. "Aggregated residual transformations for deep neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  15. Zagoruyko, Sergey, Nikos Komodakis. "Wide residual networks." arXiv preprint arXiv:1605.07146 (2016).
  16. Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, William T. Freeman. "LabelMe: a database and web-based tool for image annotation." International journal of computer vision 77.1-3 (2008): 157-173. https://doi.org/10.1007/s11263-007-0090-8