Journal Article
. 2009 Nov; 57(3):642-53.
doi: 10.1109/TBME.2009.2035305.

Computerized image-based detection and grading of lymphocytic infiltration in HER2+ breast cancer histopathology

Ajay Nagesh Basavanhally 1 Shridar Ganesan  Shannon Agner  James Peter Monaco  Michael D Feldman  John E Tomaszewski  Gyan Bhanot  Anant Madabhushi  
  • PMID: 19884074
  •     59 citations


The identification of phenotypic changes in breast cancer (BC) histopathology on account of corresponding molecular changes is of significant clinical importance in predicting disease outcome. One such example is the presence of lymphocytic infiltration (LI) in histopathology, which has been correlated with nodal metastasis and distant recurrence in HER2+ BC patients. In this paper, we present a computer-aided diagnosis (CADx) scheme to automatically detect and grade the extent of LI in digitized HER2+ BC histopathology. Lymphocytes are first automatically detected by a combination of region growing and Markov random field algorithms. Using the centers of individual detected lymphocytes as vertices, three graphs (Voronoi diagram, Delaunay triangulation, and minimum spanning tree) are constructed and a total of 50 image-derived features describing the arrangement of the lymphocytes are extracted from each sample. A nonlinear dimensionality reduction scheme, graph embedding (GE), is then used to project the high-dimensional feature vector into a reduced 3-D embedding space. A support vector machine classifier is used to discriminate samples with high and low LI in the reduced dimensional embedding space. A total of 41 HER2+ hematoxylin-and-eosin-stained images obtained from 12 patients were considered in this study. For more than 100 three-fold cross-validation trials, the architectural feature set successfully distinguished samples of high and low LI levels with a classification accuracy greater than 90%. The popular unsupervised Varma-Zisserman texton-based classification scheme was used for comparison and yielded a classification accuracy of only 60%. Additionally, the projection of the 50 image-derived features for all 41 tissue samples into a reduced dimensional space via GE allowed for the visualization of a smooth manifold that revealed a continuum between low, intermediate, and high levels of LI. Since it is known that extent of LI in BC biopsy specimens is a prognostic indicator, our CADx scheme will potentially help clinicians determine disease outcome and allow them to make better therapy recommendations for patients with HER2+ BC.

Pattern recognition software and techniques for biological image analysis.
Lior Shamir, John D Delaney, +2 authors, Ilya G Goldberg.
PLoS Comput Biol, 2010 Dec 03; 6(11). PMID: 21124870    Free PMC article.
Improving the visualization and detection of tissue folds in whole slide images through color enhancement.
Pinky A Bautista, Yukako Yagi.
J Pathol Inform, 2011 Jan 12; 1. PMID: 21221170    Free PMC article.
Computer aided diagnostic tools aim to empower rather than replace pathologists: Lessons learned from computational chess.
Jason Hipp, Thomas Flotte, +12 authors, Ulysses J Balis.
J Pathol Inform, 2011 Jul 21; 2. PMID: 21773056    Free PMC article.
An active learning based classification strategy for the minority class problem: application to histopathology annotation.
Scott Doyle, James Monaco, +2 authors, Anant Madabhushi.
BMC Bioinformatics, 2011 Nov 01; 12. PMID: 22034914    Free PMC article.
Supervised regularized canonical correlation analysis: integrating histologic and proteomic measurements for predicting biochemical recurrence following prostate surgery.
Abhishek Golugula, George Lee, +4 authors, Anant Madabhushi.
BMC Bioinformatics, 2011 Dec 21; 12. PMID: 22182303    Free PMC article.
Image microarrays (IMA): Digital pathology's missing tool.
Jason Hipp, Jerome Cheng, +15 authors, Ulysses J Balis.
J Pathol Inform, 2011 Dec 27; 2. PMID: 22200030    Free PMC article.
Automatic detection of melanoma progression by histological analysis of secondary sites.
Nikita V Orlov, Ashani T Weeraratna, +5 authors, Ilya G Goldberg.
Cytometry A, 2012 Apr 03; 81(5). PMID: 22467531    Free PMC article.
Tryggo: Old norse for truth: The real truth about ground truth: New insights into the challenges of generating ground truth maps for WSI CAD algorithm evaluation.
Jason D Hipp, Steven C Smith, +4 authors, Ulysses J Balis.
J Pathol Inform, 2012 Apr 25; 3. PMID: 22530176    Free PMC article.
Multi-field-of-view strategy for image-based outcome prediction of multi-parametric estrogen receptor-positive breast cancer histopathology: Comparison to Oncotype DX.
Ajay Basavanhally, Michael Feldman, +4 authors, Anant Madabhushi.
J Pathol Inform, 2011 Jan 01; 2. PMID: 22811953    Free PMC article.
Graphical processing unit implementation of an integrated shape-based active contour: Application to digital pathology.
Sahirzeeshan Ali, Anant Madabhushi.
J Pathol Inform, 2011 Jan 01; 2. PMID: 22811957    Free PMC article.
Cascaded discrimination of normal, abnormal, and confounder classes in histopathology: Gleason grading of prostate cancer.
Scott Doyle, Michael D Feldman, +2 authors, Anant Madabhushi.
BMC Bioinformatics, 2012 Nov 01; 13. PMID: 23110677    Free PMC article.
Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides.
Ajay Basavanhally, Shridar Ganesan, +4 authors, Anant Madabhushi.
IEEE Trans Biomed Eng, 2013 Feb 09; 60(8). PMID: 23392336    Free PMC article.
HyMaP: A hybrid magnitude-phase approach to unsupervised segmentation of tumor areas in breast cancer histology images.
Adnan M Khan, Hesham El-Daly, Emma Simmons, Nasir M Rajpoot.
J Pathol Inform, 2013 Jun 15; 4(Suppl). PMID: 23766931    Free PMC article.
Pathology imaging informatics for quantitative analysis of whole-slide images.
Sonal Kothari, John H Phan, Todd H Stokes, May D Wang.
J Am Med Inform Assoc, 2013 Aug 21; 20(6). PMID: 23959844    Free PMC article.
Highly Cited. Review.
New morphological features for grading pancreatic ductal adenocarcinomas.
Jae-Won Song, Ju-Hong Lee.
Biomed Res Int, 2013 Aug 29; 2013. PMID: 23984321    Free PMC article.
Removing batch effects from histopathological images for enhanced cancer diagnosis.
Sonal Kothari, John H Phan, +3 authors, May D Wang.
IEEE J Biomed Health Inform, 2014 May 09; 18(3). PMID: 24808220    Free PMC article.
Cancer diagnosis by nuclear morphometry using spatial information .
Hu Huang, Akif Burak Tosun, +4 authors, Gustavo K Rohde.
Pattern Recognit Lett, 2014 Jun 10; 42. PMID: 24910485    Free PMC article.
Digital Pathology: Data-Intensive Frontier in Medical Imaging: Health-information sharing, specifically of digital pathology, is the subject of this paper which discusses how sharing the rich images in pathology can stretch the capabilities of all otherwise well-practiced disciplines.
Lee A D Cooper, Alexis B Carter, +10 authors, Joel H Saltz.
Proc IEEE Inst Electr Electron Eng, 2012 Apr 01; 100(4). PMID: 25328166    Free PMC article.
Modelling the spatial heterogeneity and molecular correlates of lymphocytic infiltration in triple-negative breast cancer.
Yinyin Yuan.
J R Soc Interface, 2014 Dec 17; 12(103). PMID: 25505134    Free PMC article.
Staining correction in digital pathology by utilizing a dye amount table.
Pinky A Bautista, Yukako Yagi.
J Digit Imaging, 2015 Jan 07; 28(3). PMID: 25561073    Free PMC article.
Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology.
Andreas Heindl, Sidra Nawaz, Yinyin Yuan.
Lab Invest, 2015 Jan 20; 95(4). PMID: 25599534
Beyond immune density: critical role of spatial heterogeneity in estrogen receptor-negative breast cancer.
Sidra Nawaz, Andreas Heindl, Konrad Koelble, Yinyin Yuan.
Mod Pathol, 2015 Feb 28; 28(6). PMID: 25720324
Identifying in vivo DCE MRI markers associated with microvessel architecture and gleason grades of prostate cancer.
Asha Singanamalli, Mirabela Rusu, +7 authors, Anant Madabhushi.
J Magn Reson Imaging, 2015 Jun 26; 43(1). PMID: 26110513    Free PMC article.
Diagnostic assessment of osteosarcoma chemoresistance based on Virtual Clinical Trials.
K A Rejniak, M C Lloyd, D R Reed, M M Bui.
Med Hypotheses, 2015 Jul 02; 85(3). PMID: 26130106    Free PMC article.
Content-based image retrieval of digitized histopathology in boosted spectrally embedded spaces.
Akshay Sridhar, Scott Doyle, Anant Madabhushi.
J Pathol Inform, 2015 Jul 15; 6. PMID: 26167385    Free PMC article.
Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images.
Jun Xu, Lei Xiang, +4 authors, Anant Madabhushi.
IEEE Trans Med Imaging, 2015 Jul 25; 35(1). PMID: 26208307    Free PMC article.
Highly Cited.
Exploring automatic prostate histopathology image Gleason grading via local structure modeling.
Daihou Wang, David J Foran, +3 authors, Xin Qi.
Annu Int Conf IEEE Eng Med Biol Soc, 2016 Jan 07; 2015. PMID: 26736836    Free PMC article.
Hierarchical nucleus segmentation in digital pathology images.
Yi Gao, Vadim Ratner, +4 authors, Joel Saltz.
Proc SPIE Int Soc Opt Eng, 2016 Jul 05; 9791. PMID: 27375315    Free PMC article.
Emerging Themes in Image Informatics and Molecular Analysis for Digital Pathology.
Rohit Bhargava, Anant Madabhushi.
Annu Rev Biomed Eng, 2016 Jul 16; 18. PMID: 27420575    Free PMC article.
Spatial Heterogeneity in the Tumor Microenvironment.
Yinyin Yuan.
Cold Spring Harb Perspect Med, 2016 Aug 03; 6(8). PMID: 27481837    Free PMC article.
Automatic batch-invariant color segmentation of histological cancer images.
Sonal Kothari, John H Phan, +5 authors, May D Wang.
Proc IEEE Int Symp Biomed Imaging, 2011 Mar 01; 2011. PMID: 27532016    Free PMC article.
A supervised learning framework for pancreatic islet segmentation with multi-scale color-texture features and rolling guidance filters.
Yue Huang, Chi Liu, +2 authors, Gustavo K Rohde.
Cytometry A, 2016 Oct 22; 89(10). PMID: 27560544    Free PMC article.
Automated Segmentation of Nuclei in Breast Cancer Histopathology Images.
Maqlin Paramanandam, Michael O'Byrne, +4 authors, Vikram Pakrashi.
PLoS One, 2016 Sep 21; 11(9). PMID: 27649496    Free PMC article.
Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples.
Riku Turkki, Nina Linder, +2 authors, Johan Lundin.
J Pathol Inform, 2016 Oct 01; 7. PMID: 27688929    Free PMC article.
Multi-scale learning based segmentation of glands in digital colonrectal pathology images.
Yi Gao, William Liu, +5 authors, Allen Tannenbaum.
Proc SPIE Int Soc Opt Eng, 2016 Nov 08; 9791. PMID: 27818565    Free PMC article.
Computer-based image analysis in breast pathology.
Ziba Gandomkar, Patrick C Brennan, Claudia Mello-Thoms.
J Pathol Inform, 2017 Jan 10; 7. PMID: 28066683    Free PMC article.
An alternative reference space for H&E color normalization.
Mark D Zarella, Chan Yeoh, David E Breen, Fernando U Garcia.
PLoS One, 2017 Mar 30; 12(3). PMID: 28355298    Free PMC article.
Training a cell-level classifier for detecting basal-cell carcinoma by combining human visual attention maps with low-level handcrafted features.
Germán Corredor, Jon Whitney, +2 authors, Eduardo Romero.
J Med Imaging (Bellingham), 2017 Apr 07; 4(2). PMID: 28382314    Free PMC article.
Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent.
Angel Cruz-Roa, Hannah Gilmore, +6 authors, Anant Madabhushi.
Sci Rep, 2017 Apr 19; 7. PMID: 28418027    Free PMC article.
Highly Cited.
AIIMDs: An Integrated Framework of Automatic Idiopathic Inflammatory Myopathy Diagnosis for Muscle.
Manish Sapkota, Fujun Liu, +3 authors, Lin Yang.
IEEE J Biomed Health Inform, 2017 Apr 20; 22(3). PMID: 28422672    Free PMC article.
Towards Generation, Management, and Exploration of Combined Radiomics and Pathomics Datasets for Cancer Research.
Joel Saltz, Jonas Almeida, +6 authors, Tahsin Kurc.
AMIA Jt Summits Transl Sci Proc, 2017 Aug 18; 2017. PMID: 28815113    Free PMC article.
An Assessment of Imaging Informatics for Precision Medicine in Cancer.
C Chennubhotla, L P Clarke, +9 authors, A Sharma.
Yearb Med Inform, 2017 Oct 25; 26(1). PMID: 29063549    Free PMC article.
A Containerized Software System for Generation, Management, and Exploration of Features from Whole Slide Tissue Images.
Joel Saltz, Ashish Sharma, +9 authors, Tahsin Kurc.
Cancer Res, 2017 Nov 03; 77(21). PMID: 29092946    Free PMC article.
Improved Graph Embedding for Robust Recognition with outliers.
Peiyang Li, Weiwei Zhou, +6 authors, Peng Xu.
Sci Rep, 2018 Mar 11; 8(1). PMID: 29523793    Free PMC article.
Identifying the morphologic basis for radiomic features in distinguishing different Gleason grades of prostate cancer on MRI: Preliminary findings.
Gregory Penzias, Asha Singanamalli, +13 authors, Anant Madabhushi.
PLoS One, 2018 Sep 01; 13(8). PMID: 30169514    Free PMC article.
Machine Learning Methods for Histopathological Image Analysis.
Daisuke Komura, Shumpei Ishikawa.
Comput Struct Biotechnol J, 2018 Oct 03; 16. PMID: 30275936    Free PMC article.
Highly Cited. Review.
Stable and discriminating features are predictive of cancer presence and Gleason grade in radical prostatectomy specimens: a multi-site study.
Patrick Leo, Robin Elliott, +3 authors, Anant Madabhushi.
Sci Rep, 2018 Oct 10; 8(1). PMID: 30297720    Free PMC article.
Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology.
Kaustav Bera, Kurt A Schalper, +2 authors, Anant Madabhushi.
Nat Rev Clin Oncol, 2019 Aug 11; 16(11). PMID: 31399699    Free PMC article.
Highly Cited. Review.
Spatial Architecture and Arrangement of Tumor-Infiltrating Lymphocytes for Predicting Likelihood of Recurrence in Early-Stage Non-Small Cell Lung Cancer.
Germán Corredor, Xiangxue Wang, +9 authors, Anant Madabhushi.
Clin Cancer Res, 2018 Sep 12; 25(5). PMID: 30201760    Free PMC article.
Association of Peritumoral Radiomics With Tumor Biology and Pathologic Response to Preoperative Targeted Therapy for HER2 (ERBB2)-Positive Breast Cancer.
Nathaniel Braman, Prateek Prasanna, +17 authors, Anant Madabhushi.
JAMA Netw Open, 2019 Apr 20; 2(4). PMID: 31002322    Free PMC article.
Unsupervised Domain Adaptation for Classification of Histopathology Whole-Slide Images.
Jian Ren, Ilker Hacihaliloglu, +2 authors, Xin Qi.
Front Bioeng Biotechnol, 2019 Jun 04; 7. PMID: 31158269    Free PMC article.
Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group.
Mohamed Amgad, Elisabeth Specht Stovgaard, +66 authors, International Immuno-Oncology Biomarker Working Group.
NPJ Breast Cancer, 2020 May 16; 6. PMID: 32411818    Free PMC article.
Artificial intelligence in digital breast pathology: Techniques and applications.
Asmaa Ibrahim, Paul Gamble, +4 authors, Emad A Rakha.
Breast, 2020 Jan 15; 49. PMID: 31935669    Free PMC article.
Computer-Aided Laser Dissection: A Microdissection Workflow Leveraging Image Analysis Tools.
Jason D Hipp, Donald J Johann, +12 authors, Michael A Tangrea.
J Pathol Inform, 2019 Jan 10; 9. PMID: 30622835    Free PMC article.
Convolutional neural network initialized active contour model with adaptive ellipse fitting for nuclear segmentation on breast histopathological images.
Jun Xu, Lei Gong, +4 authors, Anant Madabhushi.
J Med Imaging (Bellingham), 2019 Mar 07; 6(1). PMID: 30840729    Free PMC article.
A novel computational method for automatic segmentation, quantification and comparative analysis of immunohistochemically labeled tissue sections.
Elena Casiraghi, Veronica Huber, +6 authors, Barbara Vergani.
BMC Bioinformatics, 2018 Oct 28; 19(Suppl 10). PMID: 30367588    Free PMC article.
Stromal cell ratio based on automated image analysis as a predictor for platinum-resistant recurrent ovarian cancer.
C Lan, J Li, +4 authors, Y Yuan.
BMC Cancer, 2019 Feb 20; 19(1). PMID: 30777045    Free PMC article.
Breast cancer outcome prediction with tumour tissue images and machine learning.
Riku Turkki, Dmitrii Byckhov, +8 authors, Nina Linder.
Breast Cancer Res Treat, 2019 May 24; 177(1). PMID: 31119567    Free PMC article.

;. PMID: 33574288