DOI QR코드

DOI QR Code

Artificial Intelligence in Pathology

  • Received : 2018.12.13
  • Accepted : 2018.12.16
  • Published : 2019.01.15

Abstract

As in other domains, artificial intelligence is becoming increasingly important in medicine. In particular, deep learning-based pattern recognition methods can advance the field of pathology by incorporating clinical, radiologic, and genomic data to accurately diagnose diseases and predict patient prognoses. In this review, we present an overview of artificial intelligence, the brief history of artificial intelligence in the medical domain, recent advances in artificial intelligence applied to pathology, and future prospects of pathology driven by artificial intelligence.

Keywords

Acknowledgement

This study was approved by the Institutional Review Board of The Catholic University of Korea Seoul St. Mary's Hospital with a waiver of informed consent (KC18SNDI0512).

References

  1. McCorduck P. Machines who think: a personal inquiry into the history and prospects of artificial intelligence. Natick: A.K. Peters, 2004.
  2. Turing AM. I. Computing machinery and intelligence. Mind 1950; 59: 433-60. https://doi.org/10.1093/mind/LIX.236.433
  3. Searle JR. Minds, brains, and programs. Behav Brain Sci 1980; 3: 417-24. https://doi.org/10.1017/S0140525X00005756
  4. Russell SJ, Norvig P. Artificial intelligence: a modern approach. Upper Saddle River: Prentice Hall, 2003.
  5. Artificial intelligence [Internet]. Wikipedia, 2018 [cited 2018 Dec 9]. Available from: https://en.wikipedia.org/wiki/Artificial_intelligence.
  6. Mortensen TL, Watt DL, Leistritz FL. Loan default prediction using logistic regression and a loan pricing model. Report No. 119549 [Internet]. Fargo: North Dakota State University, 1988 [cited 2018 Dec 7]. Available from: https://ideas.repec.org/p/ags/nddmrs/119549.html.
  7. Graham P. Better Bayesian filtering [Internet]. PAUL GRAHAM, 2003 [cited 2018 Nov 22]. Available from: http://www.paulgraham.com/better.html.
  8. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Pereira F, Burges CJ, Bottou L, Weinberger KQ, eds. Advances in neural information processing systems 25. Red Hook: Curran Associates, Inc., 2012; 1097-105.
  9. Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature 2015; 518: 529-33. https://doi.org/10.1038/nature14236
  10. Silver D, Schrittwieser J, Simonyan K, et al. Mastering the game of Go without human knowledge. Nature 2017; 550: 354-9. https://doi.org/10.1038/nature24270
  11. Hannun A, Case C, Casper J, et al. Deep speech: scaling up end-toend speech recognition [Internet]. Ithaca: arXiv, Cornell University, 2014 [cited 2018 Nov 22]. Available from: http://arxiv.org/abs/ 1412.5567.
  12. Luong MT, Pham H, Manning CD. Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015 Sep 17-21, Lisbon, Portugal. Stroudsburg: Association for Computational Linguistics, 2015; 1412-21.
  13. Wu Y, Schuster M, Chen Z, et al. Google's neural machine translation system: bridging the gap between human and machine translation [Internet]. Ithaca: arXiv, Cornell University, 2016 [cited 2018 Nov 22]. Available from: http://arxiv.org/abs/1609.08144.
  14. Antol S, Agrawal A, Lu J, et al. VQA: visual question answering. In: Proceedings of the IEEE International Conference on Computer Vision, 2015 Dec 7-13, Santiago, Chile. Washington, DC: IEEE Computer Society, 2015; 2425-33.
  15. Kim JH, Lee SW, Kwak D, et al. Multimodal residual learning for visual QA. In: Lee DD, von Luxburg U, Garnett R, et al., eds. Advances in neural information processing systems 29. Red Hook: NY Curran Associates Inc., 2016; 361-9.
  16. LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998; 86: 2278-324. https://doi.org/10.1109/5.726791
  17. Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015 Jun 7-12, Boston, MA, USA. Silver Spring: IEEE Computer Society Press, 2015; 1-9.
  18. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521: 436-44. https://doi.org/10.1038/nature14539
  19. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997; 9: 1735-80. https://doi.org/10.1162/neco.1997.9.8.1735
  20. Weizenbaum J. ELIZA: a computer program for the study of natural language communication between man and machine. Commun ACM 1966; 9: 36-45. https://doi.org/10.1145/365153.365168
  21. Shortliffe EH. Mycin: a knowledge-based computer program applied to infectious diseases. In: Proceedings of the Annual Symposium on Computer Application in Medical Care, 1977 Oct 3-5, Washington, DC, USA. New York: Institute of Electrical and Electronics Engineers, 1977; 66-9.
  22. Heckerman DE, Horvitz EJ, Nathwani BN. Toward normative expert systems: Part I. The Pathfinder project. Methods Inf Med 1992; 31: 90-105. https://doi.org/10.1055/s-0038-1634867
  23. Heckerman DE, Nathwani BN. Toward normative expert systems: Part II. Probability-based representations for efficient knowledge acquisition and inference. Methods Inf Med 1992; 31: 106-16. https://doi.org/10.1055/s-0038-1634868
  24. Vyborny CJ, Giger ML. Computer vision and artificial intelligence in mammography. AJR Am J Roentgenol 1994; 162: 699-708. https://doi.org/10.2214/ajr.162.3.8109525
  25. Kononenko I. Machine learning for medical diagnosis: history, state of the art and perspective. Artif Intell Med 2001; 23: 89-109. https://doi.org/10.1016/S0933-3657(01)00077-X
  26. Baker JA, Rosen EL, Lo JY, Gimenez EI, Walsh R, Soo MS. Computeraided detection (CAD) in screening mammography: sensitivity of commercial CAD systems for detecting architectural distortion. AJR Am J Roentgenol 2003; 181: 1083-8. https://doi.org/10.2214/ajr.181.4.1811083
  27. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016; 316: 2402-10. https://doi.org/10.1001/jama.2016.17216
  28. Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88. https://doi.org/10.1016/j.media.2017.07.005
  29. Kohli M, Prevedello LM, Filice RW, Geis JR. Implementing machine learning in radiology practice and research. AJR Am J Roentgenol 2017; 208: 754-60. https://doi.org/10.2214/AJR.16.17224
  30. Shaikhina T, Khovanova NA. Handling limited datasets with neural networks in medical applications: a small-data approach. Artif Intell Med 2017; 75: 51-63. https://doi.org/10.1016/j.artmed.2016.12.003
  31. Angermueller C, Parnamaa T, Parts L, Stegle O. Deep learning for computational biology. Mol Syst Biol 2016; 12: 878. https://doi.org/10.15252/msb.20156651
  32. Torkamani A, Andersen KG, Steinhubl SR, Topol EJ. High-definition medicine. Cell 2017; 170: 828-43. https://doi.org/10.1016/j.cell.2017.08.007
  33. Wainberg M, Merico D, Delong A, Frey BJ. Deep learning in biomedicine. Nat Biotechnol 2018; 36: 829-38. https://doi.org/10.1038/nbt.4233
  34. Xiong HY, Alipanahi B, Lee LJ, et al. RNA splicing: the human splicing code reveals new insights into the genetic determinants of disease. Science 2015; 347: 1254806. https://doi.org/10.1126/science.1254806
  35. Poplin R, Chang PC, Alexander D, et al. A universal SNP and smallindel variant caller using deep neural networks. Nat Biotechnol 2018; 36: 983-7. https://doi.org/10.1038/nbt.4235
  36. Rajkomar A, Oren E, Chen K, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med 2018; 1: 18. https://doi.org/10.1038/s41746-018-0029-1
  37. Fernandes K, Chicco D, Cardoso JS, Fernandes J. Supervised deep learning embeddings for the prediction of cervical cancer diagnosis. PeerJ Comput Sci 2018; 4: e154. https://doi.org/10.7717/peerj-cs.154
  38. Ye JJ. Artificial intelligence for pathologists is not near: it is here: description of a prototype that can transform how we practice pathology tomorrow. Arch Pathol Lab Med 2015; 139: 929-35. https://doi.org/10.5858/arpa.2014-0478-OA
  39. Beck JR, Salem DN, Estes NA, Pauker SG. A computer-based Markov decision analysis of the management of symptomatic bifascicular block: the threshold probability for pacing. J Am Coll Cardiol 1987; 9: 920-35. https://doi.org/10.1016/S0735-1097(87)80251-6
  40. Schaefer AJ, Bailey MD, Shechter SM, Roberts MS. Modeling medical treatment using Markov decision processes. In: Brandeau ML, Sainfort F, Pierskalla WP, eds. Operations research and health care: a handbook of methods and applications. Boston: Kluwer Academic Publisher, 2004; 593-612.
  41. Alagoz O, Hsu H, Schaefer AJ, Roberts MS. Markov decision processes: a tool for sequential decision making under uncertainty. Med Decis Making 2010; 30: 474-83. https://doi.org/10.1177/0272989X09353194
  42. Harbias A, Salmo E, Crump A. Implications of observer variation in Gleason scoring of prostate cancer on clinical management: a collaborative audit. Gulf J Oncolog 2017; 1: 41-5.
  43. Ozkan TA, Eruyar AT, Cebeci OO, Memik O, Ozcan L, Kuskonmaz I. Interobserver variability in Gleason histological grading of prostate cancer. Scand J Urol 2016; 50: 420-4. https://doi.org/10.1080/21681805.2016.1206619
  44. Janowczyk A, Madabhushi A. Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J Pathol Inform 2016; 7: 29. https://doi.org/10.4103/2153-3539.186902
  45. Komura D, Ishikawa S. Machine learning methods for histopathological image analysis. Comput Struct Biotechnol J 2018; 16: 34-42. https://doi.org/10.1016/j.csbj.2018.01.001
  46. Garud H, Karri SP, Sheet D, et al. High-magnification multi-views based classification of breast fine needle aspiration cytology cell samples using fusion of decisions from deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017 Jul 21-26, Honolulu, HI, USA. New York: Institute of Electrical and Electronics Engineers, 2017; 828-33.
  47. Li Y, Ping W. Cancer metastasis detection with neural conditional random field [Internet]. Ithaca: arXiv, Cornell University, 2018 [cited 2018 Nov 22]. Available from: http://arxiv.org/abs/1806.07064.
  48. Rannen Triki A, Blaschko MB, Jung YM, et al. Intraoperative margin assessment of human breast tissue in optical coherence tomography images using deep neural networks. Comput Med Imaging Graph 2018; 69: 21-32. https://doi.org/10.1016/j.compmedimag.2018.06.002
  49. Ehteshami Bejnordi B, Mullooly M, Pfeiffer RM, et al. Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies. Mod Pathol 2018; 31: 1502-12. https://doi.org/10.1038/s41379-018-0073-z
  50. Litjens G, Sanchez CI, Timofeeva N, et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep 2016; 6: 26286. https://doi.org/10.1038/srep26286
  51. Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv 2013; 16: 411-8.
  52. Teramoto A, Tsukamoto T, Kiriyama Y, Fujita H. Automated classification of lung cancer types from cytological images using deep convolutional neural networks. Biomed Res Int 2017; 2017: 4067832.
  53. Yu KH, Zhang C, Berry GJ, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun 2016; 7: 12474. https://doi.org/10.1038/ncomms12474
  54. Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 2018; 24: 1559-67. https://doi.org/10.1038/s41591-018-0177-5
  55. Campanella G, Silva VW, Fuchs TJ. Terabyte-scale deep multiple instance learning for classification and localization in pathology [Internet]. Ithaca: arXiv, Cornell University, 2018 [cited 2018 Nov 22]. Available from: http://arxiv.org/abs/1805.06983.
  56. Arvaniti E, Fricker KS, Moret M, et al. Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Sci Rep 2018; 8: 12054. https://doi.org/10.1038/s41598-018-30535-1
  57. Zhou N, Fedorov A, Fennessy F, Kikinis R, Gao Y. Large scale digital prostate pathology image analysis combining feature extraction and deep neural network [Internet]. Ithaca: arXiv, Cornell University, 2017 [cited 2018 Nov 22]. Available from: http://arxiv.org/abs/ 1705.02678.
  58. Nagpal K, Foote D, Liu Y, et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer [Internet]. Ithaca: arXiv, Cornell University, 2018 [cited 2018 Nov 22]. Available from: http://arxiv.org/abs/1811.06497.
  59. Ertosun MG, Rubin DL. Automated grading of gliomas using deep learning in digital pathology images: a modular approach with ensemble of convolutional neural networks. AMIA Annu Symp Proc 2015; 2015: 1899-908.
  60. Mobadersany P, Yousefi S, Amgad M, et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci U S A 2018; 115: E2970-E9. https://doi.org/10.1073/pnas.1717139115
  61. Wu M, Yan C, Liu H, Liu Q. Automatic classification of ovarian cancer types from cytological images using deep convolutional neural networks. Biosci Rep 2018; 38: BSR20180289. https://doi.org/10.1042/BSR20180289
  62. Zhang L, Lu L, Nogues I, Summers RM, Liu S, Yao J. DeepPap: deep convolutional networks for cervical cell classification. IEEE J Biomed Health Inform 2017; 21: 1633-43. https://doi.org/10.1109/JBHI.2017.2705583
  63. Xu M, Papageorgiou DP, Abidi SZ, Dao M, Zhao H, Karniadakis GE. A deep convolutional neural network for classification of red blood cells in sickle cell anemia. PLoS Comput Biol 2017; 13: e1005746. https://doi.org/10.1371/journal.pcbi.1005746
  64. Meier A, Nekolla K, Earle S, et al. End-to-end learning to predict survival in patients with gastric cancer using convolutional neural networks. Ann Oncol 2018; 29(Suppl 8): mdy269.075.
  65. Xie W, Noble JA, Zisserman A. Microscopy cell counting and detection with fully convolutional regression networks. Comput Methods Biomech Biomed Eng Imaging Vis 2016; 6: 283-92.
  66. Tuominen VJ, Ruotoistenmaki S, Viitanen A, Jumppanen M, Isola J. ImmunoRatio: a publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67. Breast Cancer Res 2010; 12: R56. https://doi.org/10.1186/bcr2615
  67. Meijering E. Cell segmentation: 50 years down the road [life sciences]. IEEE Signal Process Mag 2012; 29: 140-5. https://doi.org/10.1109/MSP.2012.2204190
  68. Ruifrok AC, Johnston DA. Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol 2001; 23: 291-9.
  69. Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 1979; 9: 62-6. https://doi.org/10.1109/TSMC.1979.4310076
  70. Zhang L, Sonka M, Lu L, Summers RM, Yao J. Combining fully convolutional networks and graph-based approach for automated segmentation of cervical cell nuclei. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), 2017 Apr 18-21, Melbourne, VIC, Australia. New York: Institute of Electrical and Electronics Engineers, 2017; 406-9.
  71. Chen H, Qi X, Yu L, Heng PA. DCAN: deep contour-aware networks for accurate gland segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016 Jun 27- 30, Las Vegas, NV, USA. New York: Institute of Electrical and Electronics Engineers, 2016; 2487-96.
  72. CAMELYON16 Consortium. CAMELYON16. CAMELYON16 ISBI challenge on cancer metastasis detection in lymph node, 2015 [Internet]. Grand-Challenges, 2016 [cited 2018 Nov 22]. Available from: https://camelyon16.grand-challenge.org/.
  73. The Cancer Genome Atlas [Internet]. Bethesda: The Cancer Genome Atlas, National Cancer Institute, 2011 [cited 2018 Nov 22]. Available from: https://cancergenome.nih.gov/.

Cited by

  1. Applications of deep learning for the analysis of medical data vol.42, pp.6, 2019, https://doi.org/10.1007/s12272-019-01162-9
  2. PROMISE CLIP Project: A Retrospective, Multicenter Study for Prostate Cancer that Integrates Clinical, Imaging and Pathology Data vol.9, pp.15, 2019, https://doi.org/10.3390/app9152982
  3. Key challenges for delivering clinical impact with artificial intelligence vol.17, pp.1, 2019, https://doi.org/10.1186/s12916-019-1426-2
  4. Role of artificial intelligence in diagnostic oral pathology-A modern approach vol.24, pp.1, 2020, https://doi.org/10.4103/jomfp.jomfp_215_19
  5. Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine vol.2020, 2019, https://doi.org/10.1093/database/baaa010
  6. Segmentation and Classification in Digital Pathology for Glioma Research: Challenges and Deep Learning Approaches vol.14, 2019, https://doi.org/10.3389/fnins.2020.00027
  7. Feasibility of fully automated classification of whole slide images based on deep learning vol.24, pp.1, 2020, https://doi.org/10.4196/kjpp.2020.24.1.89
  8. Current Status of Computational Intelligence Applications in Dermatological Clinical Practice vol.14, 2019, https://doi.org/10.2174/1874372202014010006
  9. Same same but different: A Web‐based deep learning application revealed classifying features for the histopathologic distinction of cortical malformations vol.61, pp.3, 2020, https://doi.org/10.1111/epi.16447
  10. Introduction to digital pathology and computer-aided pathology vol.54, pp.2, 2020, https://doi.org/10.4132/jptm.2019.12.31
  11. Artificial intelligence and the future of global health vol.395, pp.10236, 2019, https://doi.org/10.1016/s0140-6736(20)30226-9
  12. Artificial intelligence as the next step towards precision pathology vol.288, pp.1, 2019, https://doi.org/10.1111/joim.13030
  13. The future of pathology is digital vol.216, pp.9, 2019, https://doi.org/10.1016/j.prp.2020.153040
  14. The use of artificial intelligence, machine learning and deep learning in oncologic histopathology vol.49, pp.9, 2019, https://doi.org/10.1111/jop.13042
  15. Advances in tissue-based imaging: impact on oncology research and clinical practice vol.20, pp.10, 2019, https://doi.org/10.1080/14737159.2020.1770599
  16. Explainable Machine Learning Model for Predicting GI Bleed Mortality in the Intensive Care Unit vol.115, pp.10, 2020, https://doi.org/10.14309/ajg.0000000000000632
  17. Prediction of clinically actionable genetic alterations from colorectal cancer histopathology images using deep learning vol.26, pp.40, 2019, https://doi.org/10.3748/wjg.v26.i40.6207
  18. Weakly-supervised learning for lung carcinoma classification using deep learning vol.10, 2020, https://doi.org/10.1038/s41598-020-66333-x
  19. Scoring pleurisy in slaughtered pigs using convolutional neural networks vol.51, pp.1, 2020, https://doi.org/10.1186/s13567-020-00775-z
  20. Use and Control of Artificial Intelligence in Patients Across the Medical Workflow: Single-Center Questionnaire Study of Patient Perspectives vol.23, pp.2, 2019, https://doi.org/10.2196/24221
  21. A pathologist-annotated dataset for validating artificial intelligence: A project description and pilot study vol.12, pp.1, 2019, https://doi.org/10.4103/jpi.jpi_83_20
  22. Amsterdam International Consensus Meeting: tumor response scoring in the pathology assessment of resected pancreatic cancer after neoadjuvant therapy vol.34, pp.1, 2019, https://doi.org/10.1038/s41379-020-00683-9
  23. Generalizability of Deep Learning System for the Pathologic Diagnosis of Various Cancers vol.11, pp.2, 2021, https://doi.org/10.3390/app11020808
  24. Automated Classification and Segmentation in Colorectal Images Based on Self-Paced Transfer Network vol.2021, 2021, https://doi.org/10.1155/2021/6683931
  25. Artificial Intelligence in Medicine: A Multinational Multi-Center Survey on the Medical and Dental Students' Perception vol.9, 2019, https://doi.org/10.3389/fpubh.2021.795284
  26. An empirical analysis of machine learning frameworks for digital pathology in medical science vol.1767, pp.1, 2019, https://doi.org/10.1088/1742-6596/1767/1/012031
  27. Evaluating Cancer-Related Biomarkers Based on Pathological Images: A Systematic Review vol.11, 2019, https://doi.org/10.3389/fonc.2021.763527
  28. Idiosyncratic Drug-Induced Liver Injury (DILI) and Herb-Induced Liver Injury (HILI): Diagnostic Algorithm Based on the Quantitative Roussel Uclaf Causality Assessment Method (RUCAM) vol.11, pp.3, 2019, https://doi.org/10.3390/diagnostics11030458
  29. Artificial intelligence and computational pathology vol.101, pp.4, 2019, https://doi.org/10.1038/s41374-020-00514-0
  30. Sliding window based deep ensemble system for breast cancer classification vol.45, pp.4, 2021, https://doi.org/10.1080/03091902.2021.1896814
  31. Proof of Concept for a Deep Learning Algorithm for Identification and Quantification of Key Microscopic Features in the Murine Model of DSS-Induced Colitis vol.49, pp.4, 2021, https://doi.org/10.1177/0192623320987804
  32. Translational Applications of Artificial Intelligence and Machine Learning for Diagnostic Pathology in Lymphoid Neoplasms: A Comprehensive and Evolutive Analysis vol.11, pp.6, 2021, https://doi.org/10.3390/biom11060793
  33. Integrated digital pathology at scale: A solution for clinical diagnostics and cancer research at a large academic medical center vol.28, pp.9, 2019, https://doi.org/10.1093/jamia/ocab085
  34. Searching Images for Consensus : Can AI Remove Observer Variability in Pathology? vol.191, pp.10, 2021, https://doi.org/10.1016/j.ajpath.2021.01.015
  35. Deep Learning on Oral Squamous Cell Carcinoma Ex Vivo Fluorescent Confocal Microscopy Data: A Feasibility Study vol.10, pp.22, 2019, https://doi.org/10.3390/jcm10225326
  36. Deep learning-based histopathological segmentation for whole slide images of colorectal cancer in a compressed domain vol.11, pp.1, 2019, https://doi.org/10.1038/s41598-021-01905-z
  37. Development and operation of a digital platform for sharing pathology image data vol.21, pp.1, 2019, https://doi.org/10.1186/s12911-021-01466-1
  38. Advances in Digital Pathology: From Artificial Intelligence to Label-Free Imaging vol.37, pp.6, 2019, https://doi.org/10.1159/000518494
  39. Whole-slide imaging, tissue image analysis, and artificial intelligence in veterinary pathology: An updated introduction and review vol.59, pp.1, 2019, https://doi.org/10.1177/03009858211040484