Review
. 2020 Jan; 49:267-273.
doi: 10.1016/j.breast.2019.12.007.

Artificial intelligence in digital breast pathology: Techniques and applications

Asmaa Ibrahim 1 Paul Gamble 2 Ronnachai Jaroensri 2 Mohammed M Abdelsamea 3 Craig H Mermel 2 Po-Hsuan Cameron Chen 2 Emad A Rakha 4 
Affiliations
  • PMID: 31935669
  •     63 References
  •     7 citations

Abstract

Breast cancer is the most common cancer and second leading cause of cancer-related death worldwide. The mainstay of breast cancer workup is histopathological diagnosis - which guides therapy and prognosis. However, emerging knowledge about the complex nature of cancer and the availability of tailored therapies have exposed opportunities for improvements in diagnostic precision. In parallel, advances in artificial intelligence (AI) along with the growing digitization of pathology slides for the primary diagnosis are a promising approach to meet the demand for more accurate detection, classification and prediction of behaviour of breast tumours. In this article, we cover the current and prospective uses of AI in digital pathology for breast cancer, review the basics of digital pathology and AI, and outline outstanding challenges in the field.

Keywords: (Artificial intelligence); (Deep learning); (Machine learning); (Whole slide image); AI; Applications; Breast cancer; Breast pathology; DL; Digital; ML; Pathology; WSI.

Comparing subjective and digital image analysis HER2/neu expression scores with conventional and modified FISH scores in breast cancer.
I Skaland, I Øvestad, +4 authors, J P A Baak.
J Clin Pathol, 2007 Apr 07; 61(1). PMID: 17412872
Whole-Slide Image Focus Quality: Automatic Assessment and Impact on AI Cancer Detection.
Timo Kohlberger, Yun Liu, +5 authors, Martin C Stumpe.
J Pathol Inform, 2020 Jan 11; 10. PMID: 31921487    Free PMC article.
Use of Biomarkers to Guide Decisions on Adjuvant Systemic Therapy for Women With Early-Stage Invasive Breast Cancer: ASCO Clinical Practice Guideline Update-Integration of Results From TAILORx.
Fabrice Andre, Nofisat Ismaila, +10 authors, Vered Stearns.
J Clin Oncol, 2019 Jun 01; 37(22). PMID: 31150316
HER-2/neu in breast cancer: interobserver variability and performance of immunohistochemistry with 4 antibodies compared with fluorescent in situ hybridization.
T A Thomson, M M Hayes, +5 authors, R L Parker.
Mod Pathol, 2001 Nov 14; 14(11). PMID: 11706067
Assessment of HER-2/neu status in breast cancer. Automated Cellular Imaging System (ACIS)-assisted quantitation of immunohistochemical assay achieves high accuracy in comparison with fluorescence in situ hybridization assay as the standard.
S Wang, M H Saboorian, +6 authors, R Ashfaq.
Am J Clin Pathol, 2001 Oct 17; 116(4). PMID: 11601134
Diagnostic Efficiency in Digital Pathology: A Comparison of Optical Versus Digital Assessment in 510 Surgical Pathology Cases.
Anne M Mills, Sarah E Gradecki, +5 authors, Helen P Cathro.
Am J Surg Pathol, 2017 Sep 07; 42(1). PMID: 28877052
Inter-laboratory and inter-observer reproducibility of immunohistochemical assessment of the Ki-67 labelling index in a large multi-centre trial.
Michael Mengel, Reinhard von Wasielewski, +3 authors, Hans Kreipe.
J Pathol, 2002 Oct 11; 198(3). PMID: 12375261
Computerized image-based detection and grading of lymphocytic infiltration in HER2+ breast cancer histopathology.
Ajay Nagesh Basavanhally, Shridar Ganesan, +5 authors, Anant Madabhushi.
IEEE Trans Biomed Eng, 2009 Nov 04; 57(3). PMID: 19884074
Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent.
Angel Cruz-Roa, Hannah Gilmore, +6 authors, Anant Madabhushi.
Sci Rep, 2017 Apr 19; 7. PMID: 28418027    Free PMC article.
Highly Cited.
A boosted Bayesian multiresolution classifier for prostate cancer detection from digitized needle biopsies.
Scott Doyle, Michael Feldman, John Tomaszewski, Anant Madabhushi.
IEEE Trans Biomed Eng, 2010 Jun 24; 59(5). PMID: 20570758
Artificial Intelligence-Based Breast Cancer Nodal Metastasis Detection: Insights Into the Black Box for Pathologists.
Yun Liu, Timo Kohlberger, +7 authors, Martin C Stumpe.
Arch Pathol Lab Med, 2018 Oct 09; 143(7). PMID: 30295070
Highly Cited.
Improved automatic detection and segmentation of cell nuclei in histopathology images.
Yousef Al-Kofahi, Wiem Lassoued, William Lee, Badrinath Roysam.
IEEE Trans Biomed Eng, 2009 Nov 04; 57(4). PMID: 19884070
Highly Cited.
Computational pathology of pre-treatment biopsies identifies lymphocyte density as a predictor of response to neoadjuvant chemotherapy in breast cancer.
H Raza Ali, Aliakbar Dariush, +16 authors, Carlos Caldas.
Breast Cancer Res, 2016 Feb 18; 18(1). PMID: 26882907    Free PMC article.
Assessment of algorithms for mitosis detection in breast cancer histopathology images.
Mitko Veta, Paul J van Diest, +26 authors, Josien P W Pluim.
Med Image Anal, 2014 Dec 31; 20(1). PMID: 25547073
Artificial Intelligence and Digital Pathology: Challenges and Opportunities.
Hamid Reza Tizhoosh, Liron Pantanowitz.
J Pathol Inform, 2019 Jan 05; 9. PMID: 30607305    Free PMC article.
Highly Cited. Review.
Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs.
Varun Gulshan, Lily Peng, +12 authors, Dale R Webster.
JAMA, 2016 Nov 30; 316(22). PMID: 27898976
Highly Cited.
Understanding diagnostic variability in breast pathology: lessons learned from an expert consensus review panel.
Kimberly H Allison, Lisa M Reisch, +5 authors, Joann G Elmore.
Histopathology, 2014 Feb 12; 65(2). PMID: 24511905    Free PMC article.
Multi-Pass Adaptive Voting for Nuclei Detection in Histopathological Images.
Cheng Lu, Hongming Xu, +3 authors, Anant Madabhushi.
Sci Rep, 2016 Oct 04; 6. PMID: 27694950    Free PMC article.
A Systematic Analysis of Discordant Diagnoses in Digital Pathology Compared With Light Microscopy.
Bethany J Williams, Philip DaCosta, Edward Goacher, Darren Treanor.
Arch Pathol Lab Med, 2017 May 04; 141(12). PMID: 28467215
Systematic Review.
Artificial Intelligence Algorithms to Assess Hormonal Status From Tissue Microarrays in Patients With Breast Cancer.
Gil Shamai, Yoav Binenbaum, +3 authors, Ron Kimmel.
JAMA Netw Open, 2019 Jul 28; 2(7). PMID: 31348505    Free PMC article.
Deep learning.
Yann LeCun, Yoshua Bengio, Geoffrey Hinton.
Nature, 2015 May 29; 521(7553). PMID: 26017442
Highly Cited. Review.
Digital pathology for the primary diagnosis of breast histopathological specimens: an innovative validation and concordance study on digital pathology validation and training.
Bethany Jill Williams, Andrew Hanby, +3 authors, Darren Treanor.
Histopathology, 2017 Sep 25; 72(4). PMID: 28940580
Prognostic value of automatically extracted nuclear morphometric features in whole slide images of male breast cancer.
Mitko Veta, Robert Kornegoor, +4 authors, Paul J van Diest.
Mod Pathol, 2012 Aug 18; 25(12). PMID: 22899294
Opportunities and obstacles for deep learning in biology and medicine.
Travers Ching, Daniel S Himmelstein, +33 authors, Casey S Greene.
J R Soc Interface, 2018 Apr 06; 15(141). PMID: 29618526    Free PMC article.
Highly Cited. Review.
Applications and challenges of digital pathology and whole slide imaging.
C Higgins.
Biotech Histochem, 2015 May 16; 90(5). PMID: 25978139
Review.
A Deep Convolutional Neural Network for segmenting and classifying epithelial and stromal regions in histopathological images.
Jun Xu, Xiaofei Luo, +2 authors, Anant Madabhushi.
Neurocomputing, 2017 Feb 06; 191. PMID: 28154470    Free PMC article.
Highly Cited.
Image analysis and machine learning in digital pathology: Challenges and opportunities.
Anant Madabhushi, George Lee.
Med Image Anal, 2016 Jul 18; 33. PMID: 27423409    Free PMC article.
Highly Cited. Review.
Automatic nuclei segmentation in H&E stained breast cancer histopathology images.
Mitko Veta, Paul J van Diest, +3 authors, Josien P W Pluim.
PLoS One, 2013 Aug 08; 8(7). PMID: 23922958    Free PMC article.
COMPUTER-AIDED GLEASON GRADING OF PROSTATE CANCER HISTOPATHOLOGICAL IMAGES USING TEXTON FORESTS.
Parmeshwar Khurd, Claus Bahlmann, +4 authors, John V Frangioni.
Proc IEEE Int Symp Biomed Imaging, 2011 Jan 12; 14-17 April 2010. PMID: 21221421    Free PMC article.
Relevance of Spatial Heterogeneity of Immune Infiltration for Predicting Risk of Recurrence After Endocrine Therapy of ER+ Breast Cancer.
Andreas Heindl, Ivana Sestak, +3 authors, Yinyin Yuan.
J Natl Cancer Inst, 2017 Sep 02; 110(2). PMID: 28859291    Free PMC article.
Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer.
Babak Ehteshami Bejnordi, Mitko Veta, +67 authors, Rui Venâncio.
JAMA, 2017 Dec 14; 318(22). PMID: 29234806    Free PMC article.
Highly Cited.
Future-proofing pathology: the case for clinical adoption of digital pathology.
Bethany Jill Williams, David Bottoms, Darren Treanor.
J Clin Pathol, 2017 Aug 07; 70(12). PMID: 28780514
Review.
Breast Cancer Multi-classification from Histopathological Images with Structured Deep Learning Model.
Zhongyi Han, Benzheng Wei, +3 authors, Shuo Li.
Sci Rep, 2017 Jun 25; 7(1). PMID: 28646155    Free PMC article.
Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype.
Heather D Couture, Lindsay A Williams, +6 authors, Marc Niethammer.
NPJ Breast Cancer, 2018 Sep 06; 4. PMID: 30182055    Free PMC article.
Complete Digital Pathology for Routine Histopathology Diagnosis in a Multicenter Hospital Network.
Juan Antonio Retamero, Jose Aneiros-Fernandez, Raimundo G Del Moral.
Arch Pathol Lab Med, 2019 Jul 12; 144(2). PMID: 31295015
Predicting breast tumor proliferation from whole-slide images: The TUPAC16 challenge.
Mitko Veta, Yujing J Heng, +30 authors, Josien P W Pluim.
Med Image Anal, 2019 Mar 13; 54. PMID: 30861443
Quantitative nuclear histomorphometry predicts oncotype DX risk categories for early stage ER+ breast cancer.
Jon Whitney, German Corredor, +6 authors, Anant Madabhushi.
BMC Cancer, 2018 Jun 01; 18(1). PMID: 29848291    Free PMC article.
Template for reporting results of biomarker testing of specimens from patients with carcinoma of the breast.
Patrick L Fitzgibbons, Deborah A Dillon, +5 authors, Sharon Nofech-Mozes.
Arch Pathol Lab Med, 2013 Nov 19; 138(5). PMID: 24236805
Deep Learning Nuclei Detection in Digitized Histology Images by Superpixels.
Sudhir Sornapudi, Ronald Joe Stanley, +6 authors, Shelliane R Frazier.
J Pathol Inform, 2018 Apr 06; 9. PMID: 29619277    Free PMC article.
A Dataset for Breast Cancer Histopathological Image Classification.
Fabio A Spanhol, Luiz S Oliveira, Caroline Petitjean, Laurent Heutte.
IEEE Trans Biomed Eng, 2015 Nov 06; 63(7). PMID: 26540668
Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology: A Multicenter Blinded Randomized Noninferiority Study of 1992 Cases (Pivotal Study).
Sanjay Mukhopadhyay, Michael D Feldman, +31 authors, Clive R Taylor.
Am J Surg Pathol, 2017 Sep 30; 42(1). PMID: 28961557    Free PMC article.
Modelling the spatial heterogeneity and molecular correlates of lymphocytic infiltration in triple-negative breast cancer.
Yinyin Yuan.
J R Soc Interface, 2014 Dec 17; 12(103). PMID: 25505134    Free PMC article.
Expression of cell cycle markers is predictive of the response to primary systemic therapy of locally advanced breast cancer.
Tímea Tőkés, Anna-Mária Tőkés, +5 authors, Magdolna Dank.
Virchows Arch, 2016 Mar 31; 468(6). PMID: 27026269
Diagnostic concordance of breast pathologists: lessons from the National Health Service Breast Screening Programme Pathology External Quality Assurance Scheme.
Emad A Rakha, Mohamed A Ahmed, +4 authors, Ian O Ellis.
Histopathology, 2016 Dec 29; 70(4). PMID: 28028831
Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases.
Andrew Janowczyk, Anant Madabhushi.
J Pathol Inform, 2016 Aug 27; 7. PMID: 27563488    Free PMC article.
Highly Cited.
How to develop machine learning models for healthcare.
Po-Hsuan Cameron Chen, Yun Liu, Lily Peng.
Nat Mater, 2019 Apr 20; 18(5). PMID: 31000806
Breast cancer histologic grading using digital microscopy: concordance and outcome association.
Emad A Rakha, Mohamed Aleskandarani, +4 authors, Leslie W Dalton.
J Clin Pathol, 2018 Mar 15; 71(8). PMID: 29535212
Systematic analysis of breast cancer morphology uncovers stromal features associated with survival.
Andrew H Beck, Ankur R Sangoi, +6 authors, Daphne Koller.
Sci Transl Med, 2011 Nov 11; 3(108). PMID: 22072638
Highly Cited.
A whole slide image-based machine learning approach to predict ductal carcinoma in situ (DCIS) recurrence risk.
Sergey Klimov, Islam M Miligy, +8 authors, Ritu Aneja.
Breast Cancer Res, 2019 Jul 31; 21(1). PMID: 31358020    Free PMC article.
Deep learning nuclei detection: A simple approach can deliver state-of-the-art results.
Henning Höfener, André Homeyer, +3 authors, Horst K Hahn.
Comput Med Imaging Graph, 2018 Oct 05; 70. PMID: 30286333
Enterprise Implementation of Digital Pathology: Feasibility, Challenges, and Opportunities.
D J Hartman, L Pantanowitz, +3 authors, G R Lauro.
J Digit Imaging, 2017 Jan 25; 30(5). PMID: 28116576    Free PMC article.
Inter-observer variability between general pathologists and a specialist in breast pathology in the diagnosis of lobular neoplasia, columnar cell lesions, atypical ductal hyperplasia and ductal carcinoma in situ of the breast.
Douglas S Gomes, Simone S Porto, Débora Balabram, Helenice Gobbi.
Diagn Pathol, 2014 Jun 21; 9. PMID: 24948027    Free PMC article.
Novel image analysis approach for quantifying expression of nuclear proteins assessed by immunohistochemistry: application to measurement of oestrogen and progesterone receptor levels in breast cancer.
Elton Rexhepaj, Donal J Brennan, +6 authors, William M Gallagher.
Breast Cancer Res, 2008 Oct 25; 10(5). PMID: 18947395    Free PMC article.
Review of the national external quality assessment (EQA) scheme for breast pathology in the UK.
Emad A Rakha, Rachel L Bennett, +3 authors, UK National Coordinating Committee for Breast Pathology (EQA Scheme Steering Committee).
J Clin Pathol, 2016 Jul 14; 70(1). PMID: 27406051
How to Read Articles That Use Machine Learning: Users' Guides to the Medical Literature.
Yun Liu, Po-Hsuan Cameron Chen, Jonathan Krause, Lily Peng.
JAMA, 2019 Nov 13; 322(18). PMID: 31714992
Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks.
David Tellez, Maschenka Balkenhol, +10 authors, Francesco Ciompi.
IEEE Trans Med Imaging, 2018 Jul 12;. PMID: 29994086
Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer.
David F Steiner, Robert MacDonald, +6 authors, Martin C Stumpe.
Am J Surg Pathol, 2018 Oct 13; 42(12). PMID: 30312179    Free PMC article.
Highly Cited.
Digital image analysis in breast pathology-from image processing techniques to artificial intelligence.
Stephanie Robertson, Hossein Azizpour, Kevin Smith, Johan Hartman.
Transl Res, 2017 Nov 28; 194. PMID: 29175265
Review.
Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images.
Joel Saltz, Rajarsi Gupta, +14 authors, Vésteinn Thorsson.
Cell Rep, 2018 Apr 05; 23(1). PMID: 29617659    Free PMC article.
Highly Cited.
Automatic quantification of HER2 gene amplification in invasive breast cancer from chromogenic in situ hybridization whole slide images.
Md Shakhawat Hossain, Matthew G Hanna, +7 authors, Yukako Yagi.
J Med Imaging (Bellingham), 2019 Nov 26; 6(4). PMID: 31763355    Free PMC article.
The analysis of cell images.
J M Prewitt, M L Mendelsohn.
Ann N Y Acad Sci, 1966 Jan 31; 128(3). PMID: 5220765
Highly Cited.
Evaluating imaging and computer-aided detection and diagnosis devices at the FDA.
Brandon D Gallas, Heang-Ping Chan, +10 authors, Margarita L Zuley.
Acad Radiol, 2012 Feb 07; 19(4). PMID: 22306064    Free PMC article.
Automated Tubule Nuclei Quantification and Correlation with Oncotype DX risk categories in ER+ Breast Cancer Whole Slide Images.
David Romo-Bucheli, Andrew Janowczyk, +2 authors, Anant Madabhushi.
Sci Rep, 2016 Sep 08; 6. PMID: 27599752    Free PMC article.
Artificial Intelligence: Is It Armageddon for Breast Radiologists?
Lawman Chiwome, Onosetale M Okojie, +2 authors, Pousettef Hamid.
Cureus, 2020 Aug 08; 12(6). PMID: 32760624    Free PMC article.
Review.
Artificial Intelligence to Decode Cancer Mechanism: Beyond Patient Stratification for Precision Oncology.
Sandip Kumar Patel, Bhawana George, Vineeta Rai.
Front Pharmacol, 2020 Sep 10; 11. PMID: 32903628    Free PMC article.
Review.
Artificial intelligence (AI) and big data in cancer and precision oncology.
Zodwa Dlamini, Flavia Zita Francies, Rodney Hull, Rahaba Marima.
Comput Struct Biotechnol J, 2020 Oct 01; 18. PMID: 32994889    Free PMC article.
Review.
Automatic ganglion cell detection for improving the efficiency and accuracy of hirschprung disease diagnosis.
Ariel Greenberg, Asaf Aizic, +3 authors, Dov Hershkovitz.
Sci Rep, 2021 Feb 10; 11(1). PMID: 33558593    Free PMC article.
Deep Learning in Head and Neck Tumor Multiomics Diagnosis and Analysis: Review of the Literature.
Xi Wang, Bin-Bin Li.
Front Genet, 2021 Mar 02; 12. PMID: 33643386    Free PMC article.
Review.
Artificial intelligence (AI) in breast cancer care - Leveraging multidisciplinary skills to improve care.
Maria Joao Cardoso, Nehmat Houssami, Giuseppe Pozzi, Brigitte Séroussi.
Breast, 2020 Dec 15; 56. PMID: 33308879    Free PMC article.
Artificial Intelligence in Pathology: From Prototype to Product.
André Homeyer, Johannes Lotz, +5 authors, Peter Hufnagl.
J Pathol Inform, 2021 May 21; 12. PMID: 34012717    Free PMC article.
Review.