Journal Article
. 2013 Aug; 8(7):e70221.
doi: 10.1371/journal.pone.0070221.

Automatic nuclei segmentation in H&E stained breast cancer histopathology images

Mitko Veta 1 Paul J van Diest  Robert Kornegoor  André Huisman  Max A Viergever  Josien P W Pluim  
  • PMID: 23922958
  •     23 References
  •     50 citations


The introduction of fast digital slide scanners that provide whole slide images has led to a revival of interest in image analysis applications in pathology. Segmentation of cells and nuclei is an important first step towards automatic analysis of digitized microscopy images. We therefore developed an automated nuclei segmentation method that works with hematoxylin and eosin (H&E) stained breast cancer histopathology images, which represent regions of whole digital slides. The procedure can be divided into four main steps: 1) pre-processing with color unmixing and morphological operators, 2) marker-controlled watershed segmentation at multiple scales and with different markers, 3) post-processing for rejection of false regions and 4) merging of the results from multiple scales. The procedure was developed on a set of 21 breast cancer cases (subset A) and tested on a separate validation set of 18 cases (subset B). The evaluation was done in terms of both detection accuracy (sensitivity and positive predictive value) and segmentation accuracy (Dice coefficient). The mean estimated sensitivity for subset A was 0.875 (±0.092) and for subset B 0.853 (±0.077). The mean estimated positive predictive value was 0.904 (±0.075) and 0.886 (±0.069) for subsets A and B, respectively. For both subsets, the distribution of the Dice coefficients had a high peak around 0.9, with the vast majority of segmentations having values larger than 0.8.

Counting mitoses by image processing in Feulgen stained breast cancer sections: the influence of resolution.
J A Beliën, J P Baak, P J van Diest, A H van Ginkel.
Cytometry, 1997 Jun 01; 28(2). PMID: 9181303
A supervised visual model for finding regions of interest in basal cell carcinoma images.
Ricardo Gutiérrez, Francisco Gómez, Lucía Roa-Peña, Eduardo Romero.
Diagn Pathol, 2011 Mar 31; 6. PMID: 21447178    Free PMC article.
Computer-aided detection of centroblasts for follicular lymphoma grading using adaptive likelihood-based cell segmentation.
Olcay Sertel, Gerard Lozanski, Arwa Shana'ah, Metin N Gurcan.
IEEE Trans Biomed Eng, 2010 Jul 03; 57(10). PMID: 20595077    Free PMC article.
Morphological grayscale reconstruction in image analysis: applications and efficient algorithms.
L Vincent.
IEEE Trans Image Process, 1993 Jan 01; 2(2). PMID: 18296207
Highly Cited.
Expectation-maximization-driven geodesic active contour with overlap resolution (EMaGACOR): application to lymphocyte segmentation on breast cancer histopathology.
Hussain Fatakdawala, Jun Xu, +5 authors, Anant Madabhushi.
IEEE Trans Biomed Eng, 2010 Feb 23; 57(7). PMID: 20172780
Systematic random sampling for selective interactive nuclear morphometry in breast cancer sections. Refinement and multiobserver evaluation.
J C Fleege, P J van Diest, J P Baak.
Anal Quant Cytol Histol, 1993 Aug 01; 15(4). PMID: 8397649
Pathological prognostic factors in breast cancer. I. The value of histological grade in breast cancer: experience from a large study with long-term follow-up.
C W Elston, I O Ellis.
Histopathology, 1991 Nov 01; 19(5). PMID: 1757079
Highly Cited.
Quantification of histochemical staining by color deconvolution.
A C Ruifrok, D A Johnston.
Anal Quant Cytol Histol, 2001 Sep 04; 23(4). PMID: 11531144
Highly Cited.
How to measure image quality in tissue-based diagnosis (diagnostic surgical pathology).
Klaus Kayser, Jürgen Görtler, +5 authors, Gian Kayser.
Diagn Pathol, 2008 Aug 05; 3 Suppl 1. PMID: 18673499    Free PMC article.
Histopathological image analysis: a review.
Metin N Gurcan, Laura E Boucheron, +3 authors, B Yener.
IEEE Rev Biomed Eng, 2009 Jan 01; 2. PMID: 20671804    Free PMC article.
Highly Cited. Review.
Learning regions of interest from low level maps in virtual microscopy.
David Romo, Eduardo Romero, Fabio González.
Diagn Pathol, 2011 Apr 22; 6 Suppl 1. PMID: 21489193    Free PMC article.
Constrained watershed method to infer morphology of mammalian cells in microscopic images.
Nezamoddin N Kachouie, Paul Fieguth, +3 authors, Ali Khademhosseini.
Cytometry A, 2010 Sep 28; 77(12). PMID: 20872884
At convenience and systematic random sampling: effects on the prognostic value of nuclear area assessments in breast cancer patients.
I Jannink, J N Bennen, +2 authors, J P Baak.
Breast Cancer Res Treat, 1995 Jan 01; 36(1). PMID: 7579507
Prognostic value of automatically extracted nuclear morphometric features in whole slide images of male breast cancer.
Mitko Veta, Robert Kornegoor, +4 authors, Paul J van Diest.
Mod Pathol, 2012 Aug 18; 25(12). PMID: 22899294
Detection and segmentation of cell nuclei in virtual microscopy images: a minimum-model approach.
Stephan Wienert, Daniel Heim, +6 authors, Frederick Klauschen.
Sci Rep, 2012 Jul 13; 2. PMID: 22787560    Free PMC article.
Histological grading of breast carcinomas: a study of interobserver agreement.
P Robbins, S Pinder, +5 authors, C Elston.
Hum Pathol, 1995 Aug 01; 26(8). PMID: 7635449
Highly Cited.
Partitioning histopathological images: an integrated framework for supervised color-texture segmentation and cell splitting.
Hui Kong, Metin Gurcan, Kamel Belkacem-Boussaid.
IEEE Trans Med Imaging, 2011 Apr 14; 30(9). PMID: 21486712    Free PMC article.
An integrated region-, boundary-, shape-based active contour for multiple object overlap resolution in histological imagery.
Sahirzeeshan Ali, Anant Madabhushi.
IEEE Trans Med Imaging, 2012 Apr 14; 31(7). PMID: 22498689
RNAi microarray analysis in cultured mammalian cells.
Spyro Mousses, Natasha J Caplen, +9 authors, Olli Kallioniemi.
Genome Res, 2003 Oct 04; 13(10). PMID: 14525932    Free PMC article.
Theory of sampling and its application in tissue based diagnosis.
Klaus Kayser, Holger Schultz, +3 authors, Ekkehard Vollmer.
Diagn Pathol, 2009 Feb 18; 4. PMID: 19220904    Free PMC article.
Towards a computer aided diagnosis system dedicated to virtual microscopy based on stereology sampling and diffusion maps.
Philippe Belhomme, Myriam Oger, +2 authors, Paulette Herlin.
Diagn Pathol, 2011 Apr 22; 6 Suppl 1. PMID: 21489198    Free PMC article.
Spectral imaging: principles and applications.
Yuval Garini, Ian T Young, George McNamara.
Cytometry A, 2006 Sep 14; 69(8). PMID: 16969819
Applying watershed algorithms to the segmentation of clustered nuclei.
N Malpica, C O de Solórzano, +4 authors, F del Pozo.
Cytometry, 1997 Aug 01; 28(4). PMID: 9266748
Mapping stain distribution in pathology slides using whole slide imaging.
Fang-Cheng Yeh, Qing Ye, +3 authors, Chien Ho.
J Pathol Inform, 2014 Mar 29; 5(1). PMID: 24672736    Free PMC article.
Automated grading of renal cell carcinoma using whole slide imaging.
Fang-Cheng Yeh, Anil V Parwani, Liron Pantanowitz, Chien Ho.
J Pathol Inform, 2014 Sep 06; 5(1). PMID: 25191622    Free PMC article.
A computer-based automated algorithm for assessing acinar cell loss after experimental pancreatitis.
John F Eisses, Amy W Davis, +5 authors, Sohail Z Husain.
PLoS One, 2014 Oct 25; 9(10). PMID: 25343460    Free PMC article.
Adaptive localization of focus point regions via random patch probabilistic density from whole-slide, Ki-67-stained brain tumor tissue.
Yazan M Alomari, Siti Norul Huda Sheikh Abdullah, Reena Rahayu MdZin, Khairuddin Omar.
Comput Math Methods Med, 2015 Mar 21; 2015. PMID: 25793010    Free PMC article.
Castration-resistant Lgr5(+) cells are long-lived stem cells required for prostatic regeneration.
Bu-er Wang, Xi Wang, +3 authors, Melissa R Junttila.
Stem Cell Reports, 2015 May 06; 4(5). PMID: 25937372    Free PMC article.
New breast cancer prognostic factors identified by computer-aided image analysis of HE stained histopathology images.
Jia-Mei Chen, Ai-Ping Qu, +7 authors, Yan Li.
Sci Rep, 2015 May 30; 5. PMID: 26022540    Free PMC article.
A robust nonlinear tissue-component discrimination method for computational pathology.
Jacob S Sarnecki, Kathleen H Burns, +4 authors, Pei-Hsun Wu.
Lab Invest, 2016 Jan 19; 96(4). PMID: 26779829    Free PMC article.
Robust Cell Detection of Histopathological Brain Tumor Images Using Sparse Reconstruction and Adaptive Dictionary Selection.
Hai Su, Fuyong Xing, Lin Yang.
IEEE Trans Med Imaging, 2016 Jan 27; 35(6). PMID: 26812706    Free PMC article.
Hierarchical nucleus segmentation in digital pathology images.
Yi Gao, Vadim Ratner, +4 authors, Joel Saltz.
Proc SPIE Int Soc Opt Eng, 2016 Jul 05; 9791. PMID: 27375315    Free PMC article.
Automated Segmentation of Nuclei in Breast Cancer Histopathology Images.
Maqlin Paramanandam, Michael O'Byrne, +4 authors, Vikram Pakrashi.
PLoS One, 2016 Sep 21; 11(9). PMID: 27649496    Free PMC article.
A novel method for morphological pleomorphism and heterogeneity quantitative measurement: Named cell feature level co-occurrence matrix.
Akira Saito, Yasushi Numata, +5 authors, Yoichiro Yamamoto.
J Pathol Inform, 2016 Oct 01; 7. PMID: 27688927    Free PMC article.
Glioma Grading Using Cell Nuclei Morphologic Features in Digital Pathology Images.
Syed M S Reza, Khan M Iftekharuddin.
Proc SPIE Int Soc Opt Eng, 2016 Dec 13; 9785. PMID: 27942094    Free PMC article.
Computer-based image analysis in breast pathology.
Ziba Gandomkar, Patrick C Brennan, Claudia Mello-Thoms.
J Pathol Inform, 2017 Jan 10; 7. PMID: 28066683    Free PMC article.
Enhancements in localized classification for uterine cervical cancer digital histology image assessment.
Peng Guo, Haidar Almubarak, +8 authors, William V Stoecker.
J Pathol Inform, 2017 Feb 07; 7. PMID: 28163974    Free PMC article.
Cell segmentation in histopathological images with deep learning algorithms by utilizing spatial relationships.
Nuh Hatipoglu, Gokhan Bilgin.
Med Biol Eng Comput, 2017 Mar 02; 55(10). PMID: 28247185
Transcription factor Etv5 is essential for the maintenance of alveolar type II cells.
Zhen Zhang, Kim Newton, +10 authors, Vishva M Dixit.
Proc Natl Acad Sci U S A, 2017 Mar 30; 114(15). PMID: 28351980    Free PMC article.
Dual leucine zipper kinase-dependent PERK activation contributes to neuronal degeneration following insult.
Martin Larhammar, Sarah Huntwork-Rodriguez, +12 authors, Trent A Watkins.
Elife, 2017 Apr 26; 6. PMID: 28440222    Free PMC article.
αEβ7 Integrin Identifies Subsets of Pro-Inflammatory Colonic CD4+ T Lymphocytes in Ulcerative Colitis.
Christopher A Lamb, John C Mansfield, +13 authors, John A Kirby.
J Crohns Colitis, 2017 Apr 30; 11(5). PMID: 28453768    Free PMC article.
Automatic extraction of cell nuclei from H&E-stained histopathological images.
Faliu Yi, Junzhou Huang, +2 authors, Guanghua Xiao.
J Med Imaging (Bellingham), 2017 Jun 28; 4(2). PMID: 28653017    Free PMC article.
An oral cavity squamous cell carcinoma quantitative histomorphometric-based image classifier of nuclear morphology can risk stratify patients for disease-specific survival.
Cheng Lu, James S Lewis, +3 authors, Anant Madabhushi.
Mod Pathol, 2017 Aug 05; 30(12). PMID: 28776575    Free PMC article.
Automatic cellularity assessment from post-treated breast surgical specimens.
Mohammad Peikari, Sherine Salama, Sharon Nofech-Mozes, Anne L Martel.
Cytometry A, 2017 Oct 05; 91(11). PMID: 28976721    Free PMC article.
Combined MEK and ERK inhibition overcomes therapy-mediated pathway reactivation in RAS mutant tumors.
Mark Merchant, John Moffat, +20 authors, Melissa R Junttila.
PLoS One, 2017 Oct 06; 12(10). PMID: 28982154    Free PMC article.
Monitoring and Targeting Anti-VEGF Induced Hypoxia within the Viable Tumor by 19F-MRI and Multispectral Analysis.
Yunzhou Shi, Jason Oeh, +7 authors, Richard A D Carano.
Neoplasia, 2017 Oct 11; 19(11). PMID: 28987998    Free PMC article.
Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images.
Xiangxue Wang, Andrew Janowczyk, +5 authors, Anant Madabhushi.
Sci Rep, 2017 Oct 21; 7(1). PMID: 29051570    Free PMC article.
CD8+ T cell infiltration in breast and colon cancer: A histologic and statistical analysis.
James Ziai, Houston N Gilbert, +4 authors, Jeong M Kim.
PLoS One, 2018 Jan 11; 13(1). PMID: 29320521    Free PMC article.
Feature-Based Representation Improves Color Decomposition and Nuclear Detection Using a Convolutional Neural Network.
Mina Khoshdeli, Bahram Parvin.
IEEE Trans Biomed Eng, 2018 Feb 21; 65(3). PMID: 29461964    Free PMC article.
A resolution adaptive deep hierarchical (RADHicaL) learning scheme applied to nuclear segmentation of digital pathology images.
Andrew Janowczyk, Scott Doyle, Hannah Gilmore, Anant Madabhushi.
Comput Methods Biomech Biomed Eng Imaging Vis, 2018 May 08; 6(3). PMID: 29732269    Free PMC article.
A Kinect-Based Segmentation of Touching-Pigs for Real-Time Monitoring.
Miso Ju, Younchang Choi, +4 authors, Daihee Park.
Sensors (Basel), 2018 May 31; 18(6). PMID: 29843479    Free PMC article.
AlphaE Integrin Expression Is Increased in the Ileum Relative to the Colon and Unaffected by Inflammation.
Ryan Ichikawa, Christopher A Lamb, +8 authors, Mary E Keir.
J Crohns Colitis, 2018 Jun 19; 12(10). PMID: 29912405    Free PMC article.
Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers.
Cheng Lu, David Romo-Bucheli, +5 authors, Anant Madabhushi.
Lab Invest, 2018 Jul 01; 98(11). PMID: 29959421    Free PMC article.
Fusion of encoder-decoder deep networks improves delineation of multiple nuclear phenotypes.
Mina Khoshdeli, Garrett Winkelmaier, Bahram Parvin.
BMC Bioinformatics, 2018 Aug 09; 19(1). PMID: 30086715    Free PMC article.
Evaluation of nucleus segmentation in digital pathology images through large scale image synthesis.
Naiyun Zhou, Xiaxia Yu, +7 authors, Yi Gao.
Proc SPIE Int Soc Opt Eng, 2017 Feb 01; 10140. PMID: 30344361    Free PMC article.
Comparative Study on Automated Cell Nuclei Segmentation Methods for Cytology Pleural Effusion Images.
Khin Yadanar Win, Somsak Choomchuay, Kazuhiko Hamamoto, Manasanan Raveesunthornkiat.
J Healthc Eng, 2018 Oct 23; 2018. PMID: 30344991    Free PMC article.
ConvPath: A software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network.
Shidan Wang, Tao Wang, +15 authors, Guanghua Xiao.
EBioMedicine, 2019 Nov 27; 50. PMID: 31767541    Free PMC article.
Spatial Architecture and Arrangement of Tumor-Infiltrating Lymphocytes for Predicting Likelihood of Recurrence in Early-Stage Non-Small Cell Lung Cancer.
Germán Corredor, Xiangxue Wang, +9 authors, Anant Madabhushi.
Clin Cancer Res, 2018 Sep 12; 25(5). PMID: 30201760    Free PMC article.
Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study.
Jakob Nikolas Kather, Johannes Krisam, +15 authors, Niels Halama.
PLoS Med, 2019 Jan 25; 16(1). PMID: 30677016    Free PMC article.
Highly Cited.
Nuclear shape, architecture and orientation features from H&E images are able to predict recurrence in node-negative gastric adenocarcinoma.
Meng-Yao Ji, Lei Yuan, +5 authors, Wei-Guo Dong.
J Transl Med, 2019 Mar 20; 17(1). PMID: 30885234    Free PMC article.
Introduction to digital pathology and computer-aided pathology.
Soojeong Nam, Yosep Chong, +5 authors, Heounjeong Go.
J Pathol Transl Med, 2020 Feb 13; 54(2). PMID: 32045965    Free PMC article.
Multiparametric MR for non-invasive evaluation of tumour tissue histological characteristics after radionuclide therapy.
Mikael Montelius, Oscar Jalnefjord, +3 authors, Maria Ljungberg.
NMR Biomed, 2019 Jan 30; 32(3). PMID: 30693592    Free PMC article.
Artificial intelligence in digital breast pathology: Techniques and applications.
Asmaa Ibrahim, Paul Gamble, +4 authors, Emad A Rakha.
Breast, 2020 Jan 15; 49. PMID: 31935669    Free PMC article.
An automatic nuclei segmentation method based on deep convolutional neural networks for histopathology images.
Hwejin Jung, Bilal Lodhi, Jaewoo Kang.
BMC Biomed Eng, 2020 Sep 10; 1. PMID: 32903361    Free PMC article.
Convolutional neural network initialized active contour model with adaptive ellipse fitting for nuclear segmentation on breast histopathological images.
Jun Xu, Lei Gong, +4 authors, Anant Madabhushi.
J Med Imaging (Bellingham), 2019 Mar 07; 6(1). PMID: 30840729    Free PMC article.
Introduction to Digital Image Analysis in Whole-slide Imaging: A White Paper from the Digital Pathology Association.
Famke Aeffner, Mark D Zarella, +11 authors, Douglas Bowman.
J Pathol Inform, 2019 Apr 16; 10. PMID: 30984469    Free PMC article.
Highly Cited. Review.
Robust and accurate quantification of biomarkers of immune cells in lung cancer micro-environment using deep convolutional neural networks.
Lilija Aprupe, Geert Litjens, +2 authors, Niels Grabe.
PeerJ, 2019 Apr 18; 7. PMID: 30993030    Free PMC article.
Multi-Organ Gland Segmentation Using Deep Learning.
Thomas Binder, El Mehdi Tantaoui, +3 authors, Maria Gabrani.
Front Med (Lausanne), 2019 Aug 21; 6. PMID: 31428614    Free PMC article.
Nuclei-Based Features for Uterine Cervical Cancer Histology Image Analysis With Fusion-Based Classification.
Peng Guo, Koyel Banerjee, +7 authors, William V Stoecker.
IEEE J Biomed Health Inform, 2015 Nov 04; 20(6). PMID: 26529792    Free PMC article.
A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images.
Yuxin Cui, Guiying Zhang, +2 authors, Jianjun Hu.
Med Biol Eng Comput, 2019 Jul 28; 57(9). PMID: 31346949
Identifying Cross-Scale Associations between Radiomic and Pathomic Signatures of Non-Small Cell Lung Cancer Subtypes: Preliminary Results.
Charlems Alvarez-Jimenez, Alvaro A Sandino, +3 authors, Eduardo Romero.
Cancers (Basel), 2020 Dec 11; 12(12). PMID: 33297357    Free PMC article.
Genetic inactivation of RIP1 kinase does not ameliorate disease in a mouse model of ALS.
Sara Dominguez, Eugene Varfolomeev, +10 authors, Baris Bingol.
Cell Death Differ, 2020 Oct 01; 28(3). PMID: 32994544    Free PMC article.
Joint analysis of expression levels and histological images identifies genes associated with tissue morphology.
Jordan T Ash, Gregory Darnell, Daniel Munro, Barbara E Engelhardt.
Nat Commun, 2021 Mar 13; 12(1). PMID: 33707455    Free PMC article.