. 2020 Nov; 9(5):2255-2276.
doi: 10.21037/tlcr-20-591.

A narrative review of digital pathology and artificial intelligence: focusing on lung cancer

Taro Sakamoto 1 Tomoi Furukawa 1 Kris Lami 1 Hoa Hoang Ngoc Pham 1 Wataru Uegami 2 Kishio Kuroda 1 Masataka Kawai 3 Hidenori Sakanashi 4 Lee Alex Donald Cooper 5 Andrey Bychkov 2 Junya Fukuoka 1 
  • PMID: 33209648
  •     59 References
  •     4 citations


The emergence of whole slide imaging technology allows for pathology diagnosis on a computer screen. The applications of digital pathology are expanding, from supporting remote institutes suffering from a shortage of pathologists to routine use in daily diagnosis including that of lung cancer. Through practice and research large archival databases of digital pathology images have been developed that will facilitate the development of artificial intelligence (AI) methods for image analysis. Currently, several AI applications have been reported in the field of lung cancer; these include the segmentation of carcinoma foci, detection of lymph node metastasis, counting of tumor cells, and prediction of gene mutations. Although the integration of AI algorithms into clinical practice remains a significant challenge, we have implemented tumor cell count for genetic analysis, a helpful application for routine use. Our experience suggests that pathologists often overestimate the contents of tumor cells, and the use of AI-based analysis increases the accuracy and makes the tasks less tedious. However, there are several difficulties encountered in the practical use of AI in clinical diagnosis. These include the lack of sufficient annotated data for the development and validation of AI systems, the explainability of black box AI models, such as those based on deep learning that offer the most promising performance, and the difficulty in defining the ground truth data for training and validation owing to inherent ambiguity in most applications. All of these together present significant challenges in the development and clinical translation of AI methods in the practice of pathology. Additional research on these problems will help in resolving the barriers to the clinical use of AI. Helping pathologists in developing knowledge of the working and limitations of AI will benefit the use of AI in both diagnostics and research.

Keywords: Artificial intelligence; deep learning; pathology; remote diagnosis; whole slide imaging.

Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features.
Kun-Hsing Yu, Ce Zhang, +4 authors, Michael Snyder.
Nat Commun, 2016 Aug 17; 7. PMID: 27527408    Free PMC article.
Highly Cited.
Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer.
Babak Ehteshami Bejnordi, Mitko Veta, +67 authors, Rui Venâncio.
JAMA, 2017 Dec 14; 318(22). PMID: 29234806    Free PMC article.
Highly Cited.
Deep Semi Supervised Generative Learning for Automated Tumor Proportion Scoring on NSCLC Tissue Needle Biopsies.
Ansh Kapil, Armin Meier, +4 authors, Nicolas Brieu.
Sci Rep, 2018 Nov 28; 8(1). PMID: 30478349    Free PMC article.
Validation of digital pathology imaging for primary histopathological diagnosis.
David R J Snead, Yee-Wah Tsang, +16 authors, Ian A Cree.
Histopathology, 2015 Sep 27; 68(7). PMID: 26409165
Predicting cancer outcomes from histology and genomics using convolutional networks.
Pooya Mobadersany, Safoora Yousefi, +5 authors, Lee A D Cooper.
Proc Natl Acad Sci U S A, 2018 Mar 14; 115(13). PMID: 29531073    Free PMC article.
Highly Cited.
Multiplex immunofluorescence staining and image analysis assay for diffuse large B cell lymphoma.
Chung-Wein Lee, Yan J Ren, +3 authors, Suzana S Couto.
J Immunol Methods, 2019 Nov 30; 478. PMID: 31783023
Digital image analysis of multiplex fluorescence IHC in colorectal cancer recognizes the prognostic value of CDX2 and its negative correlation with SOX2.
Nair Lopes, Christian Holst Bergsland, +7 authors, Jarle Bruun.
Lab Invest, 2019 Oct 24; 100(1). PMID: 31641225    Free PMC article.
Whole-slide imaging at primary pathological diagnosis: Validation of whole-slide imaging-based primary pathological diagnosis at twelve Japanese academic institutes.
Kazuhiro Tabata, Ichiro Mori, +13 authors, Junya Fukuoka.
Pathol Int, 2017 Oct 06; 67(11). PMID: 28980740
Digital pathology in clinical use: where are we now and what is holding us back?
Jon Griffin, Darren Treanor.
Histopathology, 2016 Dec 14; 70(1). PMID: 27960232
The estimation of tumor cell percentage for molecular testing by pathologists is not accurate.
Alexander J J Smits, J Alain Kummer, +8 authors, Aryan Vink.
Mod Pathol, 2013 Jul 28; 27(2). PMID: 23887293
Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images.
Xiangxue Wang, Andrew Janowczyk, +5 authors, Anant Madabhushi.
Sci Rep, 2017 Oct 21; 7(1). PMID: 29051570    Free PMC article.
Multiplex Immunohistochemistry for Image Analysis of Tertiary Lymphoid Structures in Cancer.
Keith E Steele, Charles Brown.
Methods Mol Biol, 2018 Aug 25; 1845. PMID: 30141009
Interobserver and intraobserver variation in the morphological evaluation of noninvasive follicular thyroid neoplasm with papillary-like nuclear features in Asian practice.
Zhiyan Liu, Andrey Bychkov, +7 authors, Kennichi Kakudo.
Pathol Int, 2019 Feb 28; 69(4). PMID: 30811774
ConvPath: A software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network.
Shidan Wang, Tao Wang, +15 authors, Guanghua Xiao.
EBioMedicine, 2019 Nov 27; 50. PMID: 31767541    Free PMC article.
Mapping Driver Mutations to Histopathological Subtypes in Papillary Thyroid Carcinoma: Applying a Deep Convolutional Neural Network.
Peiling Tsou, Chang-Jiun Wu.
J Clin Med, 2019 Oct 17; 8(10). PMID: 31614962    Free PMC article.
Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning.
Nicolas Coudray, Paolo Santiago Ocampo, +6 authors, Aristotelis Tsirigos.
Nat Med, 2018 Sep 19; 24(10). PMID: 30224757
Highly Cited.
Convolutional neural networks can accurately distinguish four histologic growth patterns of lung adenocarcinoma in digital slides.
Arkadiusz Gertych, Zaneta Swiderska-Chadaj, +7 authors, Beatrice S Knudsen.
Sci Rep, 2019 Feb 08; 9(1). PMID: 30728398    Free PMC article.
Highly Cited.
Effect of neoadjuvant chemotherapy on the immune microenvironment in non-small cell lung carcinomas as determined by multiplex immunofluorescence and image analysis approaches.
Edwin R Parra, Pamela Villalobos, +19 authors, Ignacio I Wistuba.
J Immunother Cancer, 2018 Jun 07; 6(1). PMID: 29871672    Free PMC article.
The 2019 WHO classification of tumours of the digestive system.
Iris D Nagtegaal, Robert D Odze, +7 authors, WHO Classification of Tumours Editorial Board.
Histopathology, 2019 Aug 23; 76(2). PMID: 31433515    Free PMC article.
Highly Cited.
Validation of interobserver agreement in lung cancer assessment: hematoxylin-eosin diagnostic reproducibility for non-small cell lung cancer: the 2004 World Health Organization classification and therapeutically relevant subsets.
Juneko E Grilley-Olson, D Neil Hayes, +26 authors, William K Funkhouser.
Arch Pathol Lab Med, 2012 May 16; 137(1). PMID: 22583114    Free PMC article.
Reproducibility of visual estimation of lung adenocarcinoma subtype proportions.
Joanne Wright, Andrew Churg, +3 authors, Eunhee Yi.
Mod Pathol, 2019 Jun 25; 32(11). PMID: 31231130
Diagnostic agreement in the histopathological evaluation of lung cancer tissue in a population-based case-control study.
Andreas Stang, Hermann Pohlabeln, +3 authors, Karl-Heinz Jöckel.
Lung Cancer, 2006 Feb 16; 52(1). PMID: 16476504
Validating whole-slide imaging for consultation diagnoses in surgical pathology.
Thomas W Bauer, Renee J Slaw.
Arch Pathol Lab Med, 2014 May 21; 138(11). PMID: 24840034
Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer.
Jakob Nikolas Kather, Alexander T Pearson, +14 authors, Tom Luedde.
Nat Med, 2019 Jun 05; 25(7). PMID: 31160815    Free PMC article.
Highly Cited.
Validation of a whole-slide image-based teleconsultation network.
Alexi Baidoshvili, Nikolas Stathonikos, +7 authors, Paul J van Diest.
Histopathology, 2018 Jun 13; 73(5). PMID: 29893996
A grading system of lung adenocarcinomas based on histologic pattern is predictive of disease recurrence in stage I tumors.
Gabriel Sica, Akihiko Yoshizawa, +5 authors, Andre L Moreira.
Am J Surg Pathol, 2010 Jun 17; 34(8). PMID: 20551825
Highly Cited.
Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks.
Kun-Hsing Yu, Feiran Wang, +4 authors, Isaac S Kohane.
J Am Med Inform Assoc, 2020 May 05; 27(5). PMID: 32364237    Free PMC article.
Nationwide cloud-based integrated database of idiopathic interstitial pneumonias for multidisciplinary discussion.
Tomoyuki Fujisawa, Kazutaka Mori, +31 authors, Takafumi Suda.
Eur Respir J, 2019 Mar 19; 53(5). PMID: 30880283    Free PMC article.
Telepathology consultation for frozen section diagnosis in China.
Yingxin Huang, Yan Lei, +11 authors, Yongjian Deng.
Diagn Pathol, 2018 May 16; 13(1). PMID: 29759085    Free PMC article.
Noninferiority Diagnostic Value, but Also Economic and Turnaround Time Advantages From Digital Pathology.
Anna Vergani, Barbara Regis, +2 authors, Giulio Rossi.
Am J Surg Pathol, 2018 Feb 14; 42(6). PMID: 29438166
Use of dynamic telepathology utilizing a smartphone in margin control cutaneous surgery.
Patrick Emanuel, Rajan Patel, +3 authors, Mark Izzard.
ANZ J Surg, 2019 Aug 06; 89(7-8). PMID: 31379076
Assessment of Ki67 in breast cancer: recommendations from the International Ki67 in Breast Cancer working group.
Mitch Dowsett, Torsten O Nielsen, +19 authors, International Ki-67 in Breast Cancer Working Group.
J Natl Cancer Inst, 2011 Oct 01; 103(22). PMID: 21960707    Free PMC article.
Highly Cited.
Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks.
Jason W Wei, Laura J Tafe, +3 authors, Saeed Hassanpour.
Sci Rep, 2019 Mar 06; 9(1). PMID: 30833650    Free PMC article.
Highly Cited.
Pocket pathologist: A mobile application for rapid diagnostic surgical pathology consultation.
Douglas J Hartman, Anil V Parwani, +10 authors, Liron Pantanowitz.
J Pathol Inform, 2014 May 21; 5(1). PMID: 24843822    Free PMC article.
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images.
Gabriele Campanella, Matthew G Hanna, +7 authors, Thomas J Fuchs.
Nat Med, 2019 Jul 17; 25(8). PMID: 31308507    Free PMC article.
Highly Cited.
Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology.
Kaustav Bera, Kurt A Schalper, +2 authors, Anant Madabhushi.
Nat Rev Clin Oncol, 2019 Aug 11; 16(11). PMID: 31399699    Free PMC article.
Highly Cited. Review.
Automated acquisition of explainable knowledge from unannotated histopathology images.
Yoichiro Yamamoto, Toyonori Tsuzuki, +16 authors, Go Kimura.
Nat Commun, 2019 Dec 20; 10(1). PMID: 31852890    Free PMC article.
Histopathologic Assessment of Capsular Invasion in Follicular Thyroid Neoplasms-an Observer Variation Study.
Yun Zhu, Yaqiong Li, +8 authors, Andrey Bychkov.
Endocr Pathol, 2020 Apr 03; 31(2). PMID: 32236857
Telepathology and the networking of pathology diagnostic services.
R S Weinstein, K J Bloom, L S Rozek.
Arch Pathol Lab Med, 1987 Jul 01; 111(7). PMID: 3606341
Robust and accurate quantification of biomarkers of immune cells in lung cancer micro-environment using deep convolutional neural networks.
Lilija Aprupe, Geert Litjens, +2 authors, Niels Grabe.
PeerJ, 2019 Apr 18; 7. PMID: 30993030    Free PMC article.
Quantitative Image Analysis of Human Epidermal Growth Factor Receptor 2 Immunohistochemistry for Breast Cancer: Guideline From the College of American Pathologists.
Marilyn M Bui, Michael W Riben, +11 authors, M Elizabeth Hammond.
Arch Pathol Lab Med, 2019 Jan 16; 143(10). PMID: 30645156    Free PMC article.
Deep learning based tissue analysis predicts outcome in colorectal cancer.
Dmitrii Bychkov, Nina Linder, +7 authors, Johan Lundin.
Sci Rep, 2018 Feb 23; 8(1). PMID: 29467373    Free PMC article.
Highly Cited.
Social Media Use for Pathologists of All Ages.
Jerad M Gardner, Phillip H McKee.
Arch Pathol Lab Med, 2019 Mar 01; 143(3). PMID: 30816833
Problems in the reproducibility of classification of small lung adenocarcinoma: an international interobserver study.
Angela R Shih, Hironori Uruga, +8 authors, Mari Mino-Kenudson.
Histopathology, 2019 May 21; 75(5). PMID: 31107973
Complete Digital Pathology for Routine Histopathology Diagnosis in a Multicenter Hospital Network.
Juan Antonio Retamero, Jose Aneiros-Fernandez, Raimundo G Del Moral.
Arch Pathol Lab Med, 2019 Jul 12; 144(2). PMID: 31295015
Digital pathology access and usage in the UK: results from a national survey on behalf of the National Cancer Research Institute's CM-Path initiative.
Bethany Jill Williams, Jessica Lee, Karin A Oien, Darren Treanor.
J Clin Pathol, 2018 Jan 11; 71(5). PMID: 29317516    Free PMC article.
Multi-Field-of-View Deep Learning Model Predicts Nonsmall Cell Lung Cancer Programmed Death-Ligand 1 Status from Whole-Slide Hematoxylin and Eosin Images.
Lingdao Sha, Boleslaw L Osinski, +7 authors, Stephen S F Yip.
J Pathol Inform, 2019 Sep 17; 10. PMID: 31523482    Free PMC article.
Interobserver reproducibility of Gleason grading of prostatic adenocarcinoma among general pathologists.
R V Singh, S R Agashe, A V Gosavi, K R Sulhyan.
Indian J Cancer, 2012 Feb 02; 48(4). PMID: 22293266
The potential impact of reproducibility of Gleason grading in men with early stage prostate cancer managed by active surveillance: a multi-institutional study.
Jesse K McKenney, Jeff Simko, +12 authors, Canary/Early Detection Research Network Prostate Active Surveillance Study Investigators.
J Urol, 2011 Jun 18; 186(2). PMID: 21679996
Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology: A Multicenter Blinded Randomized Noninferiority Study of 1992 Cases (Pivotal Study).
Sanjay Mukhopadhyay, Michael D Feldman, +31 authors, Clive R Taylor.
Am J Surg Pathol, 2017 Sep 30; 42(1). PMID: 28961557    Free PMC article.
Similar image search for histopathology: SMILY.
Narayan Hegde, Jason D Hipp, +11 authors, Martin C Stumpe.
NPJ Digit Med, 2019 Jul 16; 2. PMID: 31304402    Free PMC article.
Automated image analysis of NSCLC biopsies to predict response to anti-PD-L1 therapy.
Sonja Althammer, Tze Heng Tan, +14 authors, Keith E Steele.
J Immunother Cancer, 2019 May 08; 7(1). PMID: 31060602    Free PMC article.
Comprehensive analysis of lung cancer pathology images to discover tumor shape and boundary features that predict survival outcome.
Shidan Wang, Alyssa Chen, +5 authors, Guanghua Xiao.
Sci Rep, 2018 Jul 12; 8(1). PMID: 29991684    Free PMC article.
Microvessel prediction in H&E Stained Pathology Images using fully convolutional neural networks.
Faliu Yi, Lin Yang, +4 authors, Guanghua Xiao.
BMC Bioinformatics, 2018 Feb 28; 19(1). PMID: 29482496    Free PMC article.
Weakly Supervised Deep Learning for Whole Slide Lung Cancer Image Analysis.
Xi Wang, Hao Chen, +6 authors, Pheng-Ann Heng.
IEEE Trans Cybern, 2019 Sep 05; 50(9). PMID: 31484154
From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge.
Peter Bandi, Oscar Geessink, +33 authors, Geert Litjens.
IEEE Trans Med Imaging, 2019 Feb 05; 38(2). PMID: 30716025
Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center.
Liron Pantanowitz, John H Sinard, +8 authors, College of American Pathologists Pathology and Laboratory Quality Center.
Arch Pathol Lab Med, 2013 May 03; 137(12). PMID: 23634907    Free PMC article.
Highly Cited. Review.
PAGE-Net: Interpretable and Integrative Deep Learning for Survival Analysis Using Histopathological Images and Genomic Data.
Jie Hao, Sai Chandra Kosaraju, +2 authors, Mingon Kang.
Pac Symp Biocomput, 2019 Dec 05; 25. PMID: 31797610
Detection of Lung Cancer Lymph Node Metastases from Whole-Slide Histopathologic Images Using a Two-Step Deep Learning Approach.
Hoa Hoang Ngoc Pham, Mitsuru Futakuchi, +3 authors, Junya Fukuoka.
Am J Pathol, 2019 Sep 22; 189(12). PMID: 31541645
RPB5-mediating protein promotes the progression of non-small cell lung cancer by regulating the proliferation and invasion.
Yu Feng, Ke Chen, +5 authors, Shaomu Chen.
J Thorac Dis, 2021 Feb 12; 13(1). PMID: 33569210    Free PMC article.
Artificial intelligence-based analysis for immunohistochemistry staining of immune checkpoints to predict resected non-small cell lung cancer survival and relapse.
Haoyue Guo, Li Diao, +7 authors, Fred R Hirsch.
Transl Lung Cancer Res, 2021 Jul 24; 10(6). PMID: 34295654    Free PMC article.
Artificial Intelligence in Digital Pathology: What Is the Future? Part 1: From the Digital Slide Onwards.
Maria Rosaria Giovagnoli, Daniele Giansanti.
Healthcare (Basel), 2021 Aug 07; 9(7). PMID: 34356236    Free PMC article.
Deep Learning Fast Screening Approach on Cytological Whole Slides for Thyroid Cancer Diagnosis.
Yi-Jia Lin, Tai-Kuang Chao, +4 authors, Ching-Wei Wang.
Cancers (Basel), 2021 Aug 08; 13(15). PMID: 34359792    Free PMC article.