. 2019 Jan; 9:38.
doi: 10.4103/jpi.jpi_53_18.

Artificial Intelligence and Digital Pathology: Challenges and Opportunities

Hamid Reza Tizhoosh 1 Liron Pantanowitz 2 
  • PMID: 30607305
  •     33 References
  •     39 citations


In light of the recent success of artificial intelligence (AI) in computer vision applications, many researchers and physicians expect that AI would be able to assist in many tasks in digital pathology. Although opportunities are both manifest and tangible, there are clearly many challenges that need to be overcome in order to exploit the AI potentials in computational pathology. In this paper, we strive to provide a realistic account of all challenges and opportunities of adopting AI algorithms in digital pathology from both engineering and pathology perspectives.

Keywords: Artificial intelligence; deep learning; digital pathology.

How does a pathologist make a diagnosis?
Gil Patrus Pena, José de Souza Andrade-Filho.
Arch Pathol Lab Med, 2009 Jan 07; 133(1). PMID: 19123724
Histopathological image analysis: a review.
Metin N Gurcan, Laura E Boucheron, +3 authors, B Yener.
IEEE Rev Biomed Eng, 2009 Jan 01; 2. PMID: 20671804    Free PMC article.
Highly Cited. Review.
Review of the current state of whole slide imaging in pathology.
Liron Pantanowitz, Paul N Valenstein, +5 authors, Terence J Colgan.
J Pathol Inform, 2011 Sep 03; 2. PMID: 21886892    Free PMC article.
Highly Cited.
Systematic analysis of breast cancer morphology uncovers stromal features associated with survival.
Andrew H Beck, Ankur R Sangoi, +6 authors, Daphne Koller.
Sci Transl Med, 2011 Nov 11; 3(108). PMID: 22072638
Highly Cited.
Artefacts: a diagnostic dilemma - a review.
Varun Rastogi, Naveen Puri, +3 authors, Rachna Sharma.
J Clin Diagn Res, 2013 Dec 04; 7(10). PMID: 24298546    Free PMC article.
Interobserver variability by pathologists in the distinction between cellular fibroadenomas and phyllodes tumors.
Thomas J Lawton, Geza Acs, +8 authors, Carol Reynolds.
Int J Surg Pathol, 2014 Aug 28; 22(8). PMID: 25161205    Free PMC article.
Artefacts in histopathology.
Shailja Chatterjee.
J Oral Maxillofac Pathol, 2014 Nov 05; 18(Suppl 1). PMID: 25364159    Free PMC article.
Diagnostic concordance among pathologists interpreting breast biopsy specimens.
Joann G Elmore, Gary M Longton, +9 authors, Donald L Weaver.
JAMA, 2015 Mar 18; 313(11). PMID: 25781441    Free PMC article.
Highly Cited.
An unsupervised feature learning framework for basal cell carcinoma image analysis.
John Arevalo, Angel Cruz-Roa, +2 authors, Fabio A González.
Artif Intell Med, 2015 May 16; 64(2). PMID: 25976208
Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features.
Haibo Wang, Angel Cruz-Roa, +6 authors, Anant Madabhushi.
J Med Imaging (Bellingham), 2015 Jul 15; 1(3). PMID: 26158062    Free PMC article.
Efficient Execution of Microscopy Image Analysis on CPU, GPU, and MIC Equipped Cluster Systems.
G Andrade, R Ferreira, +3 authors, Tahsin Kurc.
Proc Symp Comput Archit High Perform Comput, 2014 Oct 01; 2014. PMID: 26640423    Free PMC article.
The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary.
David N Louis, Arie Perry, +7 authors, David W Ellison.
Acta Neuropathol, 2016 May 10; 131(6). PMID: 27157931
Highly Cited. Review.
Passing the Turing Test Does Not Mean the End of Humanity.
Kevin Warwick, Huma Shah.
Cognit Comput, 2016 Jun 04; 8. PMID: 27257441    Free PMC article.
Image analysis and machine learning in digital pathology: Challenges and opportunities.
Anant Madabhushi, George Lee.
Med Image Anal, 2016 Jul 18; 33. PMID: 27423409    Free PMC article.
Highly Cited. Review.
Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases.
Andrew Janowczyk, Anant Madabhushi.
J Pathol Inform, 2016 Aug 27; 7. PMID: 27563488    Free PMC article.
Highly Cited.
Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification.
Le Hou, Dimitris Samaras, +3 authors, Joel H Saltz.
Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 2016 Nov 01; 2016. PMID: 27795661    Free PMC article.
Highly Cited.
Adapting to Artificial Intelligence: Radiologists and Pathologists as Information Specialists.
Saurabh Jha, Eric J Topol.
JAMA, 2016 Nov 30; 316(22). PMID: 27898975
Highly Cited.
Dermatologist-level classification of skin cancer with deep neural networks.
Andre Esteva, Brett Kuprel, +4 authors, Sebastian Thrun.
Nature, 2017 Jan 25; 542(7639). PMID: 28117445
Highly Cited.
Deep Learning in Medical Image Analysis.
Dinggang Shen, Guorong Wu, Heung-Il Suk.
Annu Rev Biomed Eng, 2017 Mar 17; 19. PMID: 28301734    Free PMC article.
Highly Cited. Review.
AlphaGo, Deep Learning, and the Future of the Human Microscopist.
Scott R Granter, Andrew H Beck, David J Papke.
Arch Pathol Lab Med, 2017 Apr 28; 141(5). PMID: 28447900
Artificial Intelligence and the Pathologist: Future Frenemies?
Gaurav Sharma, Alexis Carter.
Arch Pathol Lab Med, 2017 Apr 28; 141(5). PMID: 28447905
Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features.
Yan Xu, Zhipeng Jia, +4 authors, Eric I-Chao Chang.
BMC Bioinformatics, 2017 May 28; 18(1). PMID: 28549410    Free PMC article.
A survey on deep learning in medical image analysis.
Geert Litjens, Thijs Kooi, +6 authors, Clara I Sánchez.
Med Image Anal, 2017 Aug 05; 42. PMID: 28778026
Highly Cited. Review.
Data augmentation-assisted deep learning of hand-drawn partially colored sketches for visual search.
Jamil Ahmad, Khan Muhammad, Sung Wook Baik.
PLoS One, 2017 Sep 01; 12(8). PMID: 28859140    Free PMC article.
Digital image analysis in breast pathology-from image processing techniques to artificial intelligence.
Stephanie Robertson, Hossein Azizpour, Kevin Smith, Johan Hartman.
Transl Res, 2017 Nov 28; 194. PMID: 29175265
Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer.
Babak Ehteshami Bejnordi, Mitko Veta, +67 authors, Rui Venâncio.
JAMA, 2017 Dec 14; 318(22). PMID: 29234806    Free PMC article.
Highly Cited.
Predicting cancer outcomes from histology and genomics using convolutional networks.
Pooya Mobadersany, Safoora Yousefi, +5 authors, Lee A D Cooper.
Proc Natl Acad Sci U S A, 2018 Mar 14; 115(13). PMID: 29531073    Free PMC article.
Highly Cited.
Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images.
Joel Saltz, Rajarsi Gupta, +14 authors, Vésteinn Thorsson.
Cell Rep, 2018 Apr 05; 23(1). PMID: 29617659    Free PMC article.
Highly Cited.
Is pathology prepared for the adoption of artificial intelligence?
Stephen T C Wong.
Cancer Cytopathol, 2018 Apr 18; 126(6). PMID: 29663732
Imaging, Health Record, and Artificial Intelligence: Hype or Hope?
Marco Mazzanti, Ervina Shirka, Hortensia Gjergo, Endri Hasimi.
Curr Cardiol Rep, 2018 May 12; 20(6). PMID: 29749590
Challenges in Pathologic Staging of Renal Cell Carcinoma: A Study of Interobserver Variability Among Urologic Pathologists.
Sean R Williamson, Priya Rao, +28 authors, Kiril Trpkov.
Am J Surg Pathol, 2018 Jun 08; 42(9). PMID: 29878933
Histopathological Whole Slide Image Analysis Using Context-Based CBIR.
Yushan Zheng, Zhiguo Jiang, +4 authors, Yu Zhao.
IEEE Trans Med Imaging, 2018 Jul 04; 37(7). PMID: 29969415
Representing Medical Images With Encoded Local Projections.
Hamid R Tizhoosh, Morteza Babaie.
IEEE Trans Biomed Eng, 2018 Jul 12; 65(10). PMID: 29993412
Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the Digital Pathology Association.
Esther Abels, Liron Pantanowitz, +9 authors, Cleopatra Kozlowski.
J Pathol, 2019 Jul 30; 249(3). PMID: 31355445    Free PMC article.
Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology.
Kaustav Bera, Kurt A Schalper, +2 authors, Anant Madabhushi.
Nat Rev Clin Oncol, 2019 Aug 11; 16(11). PMID: 31399699    Free PMC article.
Highly Cited. Review.
Connecting Histopathology Imaging and Proteomics in Kidney Cancer through Machine Learning.
Francisco Azuaje, Sang-Yoon Kim, Daniel Perez Hernandez, Gunnar Dittmar.
J Clin Med, 2019 Sep 29; 8(10). PMID: 31557788    Free PMC article.
Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence.
Shivam Kalra, H R Tizhoosh, +8 authors, Liron Pantanowitz.
NPJ Digit Med, 2020 Mar 21; 3. PMID: 32195366    Free PMC article.
Introduction to digital pathology and computer-aided pathology.
Soojeong Nam, Yosep Chong, +5 authors, Heounjeong Go.
J Pathol Transl Med, 2020 Feb 13; 54(2). PMID: 32045965    Free PMC article.
Emerging role of deep learning-based artificial intelligence in tumor pathology.
Yahui Jiang, Meng Yang, +2 authors, Yan Sun.
Cancer Commun (Lond), 2020 Apr 12; 40(4). PMID: 32277744    Free PMC article.
Value of Public Challenges for the Development of Pathology Deep Learning Algorithms.
Douglas Joseph Hartman, Jeroen A W M Van Der Laak, Metin N Gurcan, Liron Pantanowitz.
J Pathol Inform, 2020 Apr 23; 11. PMID: 32318315    Free PMC article.
Artificial intelligence driven next-generation renal histomorphometry.
Briana A Santo, Avi Z Rosenberg, Pinaki Sarder.
Curr Opin Nephrol Hypertens, 2020 Mar 25; 29(3). PMID: 32205581    Free PMC article.
Role of artificial intelligence in diagnostic oral pathology-A modern approach.
Ayinampudi Bhargavi Krishna, Azra Tanveer, Pancha Venkat Bhagirath, Ashalata Gannepalli.
J Oral Maxillofac Pathol, 2020 Jun 09; 24(1). PMID: 32508465    Free PMC article.
Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning.
Mingyu Chen, Bin Zhang, +6 authors, Xiujun Cai.
NPJ Precis Oncol, 2020 Jun 19; 4. PMID: 32550270    Free PMC article.
Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses.
Liron Pantanowitz, Douglas Hartman, +8 authors, Soo Youn Cho.
Diagn Pathol, 2020 Jul 06; 15(1). PMID: 32622359    Free PMC article.
Use of AI-based tools for healthcare purposes: a survey study from consumers' perspectives.
Pouyan Esmaeilzadeh.
BMC Med Inform Decis Mak, 2020 Jul 24; 20(1). PMID: 32698869    Free PMC article.
Artificial intelligence in digital breast pathology: Techniques and applications.
Asmaa Ibrahim, Paul Gamble, +4 authors, Emad A Rakha.
Breast, 2020 Jan 15; 49. PMID: 31935669    Free PMC article.
New additional scoring formula on the Pathological Features in Stage I Lung Adenocarcinoma Patients: Impact on Survival.
Shilei Zhao, Fengzhou Li, +5 authors, Hidetaka Uramoto.
Int J Med Sci, 2020 Aug 14; 17(13). PMID: 32788866    Free PMC article.
Biobanking for glomerular diseases: a study design and protocol for KOrea Renal biobank NEtwoRk System TOward NExt-generation analysis (KORNERSTONE).
Eunjeong Kang, Yaerim Kim, +24 authors, KORNERSTONE Study Group.
BMC Nephrol, 2020 Aug 28; 21(1). PMID: 32842999    Free PMC article.
Interactive thyroid whole slide image diagnostic system using deep representation.
Pingjun Chen, Xiaoshuang Shi, +3 authors, Paul D Gader.
Comput Methods Programs Biomed, 2020 Jul 08; 195. PMID: 32634647    Free PMC article.
Artificial Intelligence in Clinical Health Care Applications: Viewpoint.
Michael van Hartskamp, Sergio Consoli, +2 authors, Anja van de Stolpe.
Interact J Med Res, 2019 Apr 06; 8(2). PMID: 30950806    Free PMC article.
Whole-Section Tumor Micro-Architecture Analysis by a Two-Dimensional Phasor-Based Approach Applied to Polarization-Dependent Second Harmonic Imaging.
Riccardo Scodellaro, Margaux Bouzin, +5 authors, Laura Sironi.
Front Oncol, 2019 Jul 06; 9. PMID: 31275857    Free PMC article.
Artificial intelligence (AI) and big data in cancer and precision oncology.
Zodwa Dlamini, Flavia Zita Francies, Rodney Hull, Rahaba Marima.
Comput Struct Biotechnol J, 2020 Oct 01; 18. PMID: 32994889    Free PMC article.
A bird's-eye view of deep learning in bioimage analysis.
Erik Meijering.
Comput Struct Biotechnol J, 2020 Oct 01; 18. PMID: 32994890    Free PMC article.
TissueWand, a Rapid Histopathology Annotation Tool.
Martin Lindvall, Alexander Sanner, +4 authors, Jonas Löwgren.
J Pathol Inform, 2020 Oct 13; 11. PMID: 33042606    Free PMC article.
Use of Artificial Intelligence in Dermatology.
Abhishek De, Aarti Sarda, Sachi Gupta, Sudip Das.
Indian J Dermatol, 2020 Nov 10; 65(5). PMID: 33165383    Free PMC article.
Attitudes Toward Artificial Intelligence Within Dermatopathology: An International Online Survey.
Sam Polesie, Phillip H McKee, +4 authors, John Paoli.
Front Med (Lausanne), 2020 Nov 17; 7. PMID: 33195357    Free PMC article.
Histo-ELISA technique for quantification and localization of tissue components.
Zhongmin Li, Silvia Goebel, Andreas Reimann, Martin Ungerer.
Sci Rep, 2020 Nov 18; 10(1). PMID: 33199754    Free PMC article.
Identifying transcriptomic correlates of histology using deep learning.
Liviu Badea, Emil Stănescu.
PLoS One, 2020 Nov 26; 15(11). PMID: 33237966    Free PMC article.
SITC cancer immunotherapy resource document: a compass in the land of biomarker discovery.
Siwen Hu-Lieskovan, Srabani Bhaumik, +14 authors, Holden T Maecker.
J Immunother Cancer, 2020 Dec 04; 8(2). PMID: 33268350    Free PMC article.
The Texas Society of Pathologists: molded by the legacy of pathology and focused on excellence in medicine for 100 years and beyond.
L Maximilian Buja.
Proc (Bayl Univ Med Cent), 2021 Jan 19; 34(1). PMID: 33456200    Free PMC article.
Artificial intelligence and computational pathology.
Miao Cui, David Y Zhang.
Lab Invest, 2021 Jan 18; 101(4). PMID: 33454724    Free PMC article.
Role of digital pathology in diagnostic histopathology in the response to COVID-19: results from a survey of experience in a UK tertiary referral hospital.
Lisa Browning, Eve Fryer, +4 authors, Clare Verrill.
J Clin Pathol, 2020 Jul 04; 74(2). PMID: 32616541    Free PMC article.
Hidden Variables in Deep Learning Digital Pathology and Their Potential to Cause Batch Effects: Prediction Model Study.
Max Schmitt, Roman Christoph Maron, +14 authors, Titus Josef Brinker.
J Med Internet Res, 2021 Feb 03; 23(2). PMID: 33528370    Free PMC article.
Multiplexed Immunohistochemistry and Digital Pathology as the Foundation for Next-Generation Pathology in Melanoma: Methodological Comparison and Future Clinical Applications.
Yannick Van Herck, Asier Antoranz, +4 authors, Francesca Maria Bosisio.
Front Oncol, 2021 Apr 16; 11. PMID: 33854972    Free PMC article.
Automated Quantification of Chronic Changes in the Kidney Biopsy: Another Step in the Right Direction.
Jeffrey B Hodgin, Laura H Mariani.
J Am Soc Nephrol, 2021 Mar 10;. PMID: 33685977    Free PMC article.
A review of microscopic analysis of blood cells for disease detection with AI perspective.
Nilkanth Mukund Deshpande, Shilpa Gite, Rajanikanth Aluvalu.
PeerJ Comput Sci, 2021 May 14; 7. PMID: 33981834    Free PMC article.
SPIE-AAPM-NCI BreastPathQ challenge: an image analysis challenge for quantitative tumor cellularity assessment in breast cancer histology images following neoadjuvant treatment.
Nicholas Petrick, Shazia Akbar, +7 authors, BreastPathQ Challenge Group.
J Med Imaging (Bellingham), 2021 May 15; 8(3). PMID: 33987451    Free PMC article.
The Value of Artificial Intelligence in Laboratory Medicine.
Ketan Paranjape, Michiel Schinkel, +5 authors, Prabath Nanayakkara.
Am J Clin Pathol, 2020 Dec 15; 155(6). PMID: 33313667    Free PMC article.
Artificial Intelligence in Pathology: From Prototype to Product.
André Homeyer, Johannes Lotz, +5 authors, Peter Hufnagl.
J Pathol Inform, 2021 May 21; 12. PMID: 34012717    Free PMC article.
What makes AI 'intelligent' and 'caring'? Exploring affect and relationality across three sites of intelligence and care.
Giulia De Togni, Sonja Erikainen, Sarah Chan, Sarah Cunningham-Burley.
Soc Sci Med, 2021 Apr 27; 277. PMID: 33901725    Free PMC article.
AI applications in renal pathology.
Yuankai Huo, Ruining Deng, +2 authors, Haichun Yang.
Kidney Int, 2021 Feb 14; 99(6). PMID: 33581198    Free PMC article.
A generalized deep learning framework for whole-slide image segmentation and analysis.
Mahendra Khened, Avinash Kori, +2 authors, Balaji Srinivasan.
Sci Rep, 2021 Jun 04; 11(1). PMID: 34078928    Free PMC article.