Journal Article
. 2015 Jan; 2015:1899-908.

Automated Grading of Gliomas using Deep Learning in Digital Pathology Images: A modular approach with ensemble of convolutional neural networks

Mehmet Günhan Ertosun 1 Daniel L Rubin 2 
  • PMID: 26958289
  •     13 References
  •     56 citations


Brain glioma is the most common primary malignant brain tumors in adults with different pathologic subtypes: Lower Grade Glioma (LGG) Grade II, Lower Grade Glioma (LGG) Grade III, and Glioblastoma Multiforme (GBM) Grade IV. The survival and treatment options are highly dependent of this glioma grade. We propose a deep learning-based, modular classification pipeline for automated grading of gliomas using digital pathology images. Whole tissue digitized images of pathology slides obtained from The Cancer Genome Atlas (TCGA) were used to train our deep learning modules. Our modular pipeline provides diagnostic quality statistics, such as precision, sensitivity and specificity, of the individual deep learning modules, and (1) facilitates training given the limited data in this domain, (2) enables exploration of different deep learning structures for each module, (3) leads to developing less complex modules that are simpler to analyze, and (4) provides flexibility, permitting use of single modules within the framework or use of other modeling or machine learning applications, such as probabilistic graphical models or support vector machines. Our modular approach helps us meet the requirements of minimum accuracy levels that are demanded by the context of different decision points within a multi-class classification scheme. Convolutional Neural Networks are trained for each module for each sub-task with more than 90% classification accuracies on validation data set, and achieved classification accuracy of 96% for the task of GBM vs LGG classification, 71% for further identifying the grade of LGG into Grade II or Grade III on independent data set coming from new patients from the multi-institutional repository.

Other Links

Free PMC article 
Comparison of effects of socioeconomic and geographic variations on survival for adults and children with glioma.
Ming-Yuan Tseng, Jen-Ho Tseng, Edwin Merchant.
J Neurosurg, 2007 Mar 03; 105(4 Suppl). PMID: 17328280
The 2007 WHO classification of tumours of the central nervous system.
David N Louis, Hiroko Ohgaki, +5 authors, Paul Kleihues.
Acta Neuropathol, 2007 Jul 10; 114(2). PMID: 17618441    Free PMC article.
Highly Cited. Review.
In silico analysis of nuclei in glioblastoma using large-scale microscopy images improves prediction of treatment response.
Jun Kong, Lee Cooper, +4 authors, Daniel Brat.
Annu Int Conf IEEE Eng Med Biol Soc, 2012 Jan 19; 2011. PMID: 22254257    Free PMC article.
The epidemiology of glioma in adults: a "state of the science" review.
Quinn T Ostrom, Luc Bauchet, +9 authors, Jill S Barnholtz-Sloan.
Neuro Oncol, 2014 May 21; 16(7). PMID: 24842956    Free PMC article.
Highly Cited. Review.
Epidemiology of glial and non-glial brain tumours in Europe.
Emanuele Crocetti, Annalisa Trama, +8 authors, RARECARE working group.
Eur J Cancer, 2012 Jan 10; 48(10). PMID: 22227039
Highly Cited.
Population-based survival data for brain tumors in Korea.
Kyu-Won Jung, Heon Yoo, +3 authors, Seung Hoon Lee.
J Neurooncol, 2012 Jun 05; 109(2). PMID: 22660961
Improving diagnostic accuracy and interobserver concordance in the classification and grading of primary gliomas.
S W Coons, P C Johnson, +2 authors, D K Pearl.
Cancer, 1997 Apr 01; 79(7). PMID: 9083161
Highly Cited.
CBTRUS statistical report: Primary brain and central nervous system tumors diagnosed in the United States in 2006-2010.
Quinn T Ostrom, Haley Gittleman, +6 authors, Jill S Barnholtz-Sloan.
Neuro Oncol, 2013 Oct 30; 15 Suppl 2. PMID: 24137015    Free PMC article.
Highly Cited.
Machine-based morphologic analysis of glioblastoma using whole-slide pathology images uncovers clinically relevant molecular correlates.
Jun Kong, Lee A D Cooper, +8 authors, Daniel J Brat.
PLoS One, 2013 Nov 16; 8(11). PMID: 24236209    Free PMC article.
Survival of European patients with central nervous system tumors.
Milena Sant, Pamela Minicozzi, +4 authors, EUROCARE Working Group.
Int J Cancer, 2011 Aug 02; 131(1). PMID: 21805473
Grading of astrocytomas. A simple and reproducible method.
C Daumas-Duport, B Scheithauer, J O'Fallon, P Kelly.
Cancer, 1988 Nov 15; 62(10). PMID: 3179928
Highly Cited.
Grading of brain tumours. The British experience.
R O Weller.
Neurosurg Rev, 1992 Jan 01; 15(1). PMID: 1316572
Pathology imaging informatics for quantitative analysis of whole-slide images.
Sonal Kothari, John H Phan, Todd H Stokes, May D Wang.
J Am Med Inform Assoc, 2013 Aug 21; 20(6). PMID: 23959844    Free PMC article.
Highly Cited. Review.
Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis.
Geert Litjens, Clara I Sánchez, +7 authors, Jeroen van der Laak.
Sci Rep, 2016 May 24; 6. PMID: 27212078    Free PMC article.
Highly Cited.
Image analysis and machine learning in digital pathology: Challenges and opportunities.
Anant Madabhushi, George Lee.
Med Image Anal, 2016 Jul 18; 33. PMID: 27423409    Free PMC article.
Highly Cited. Review.
Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases.
Andrew Janowczyk, Anant Madabhushi.
J Pathol Inform, 2016 Aug 27; 7. PMID: 27563488    Free PMC article.
Highly Cited.
Identification of Histological Correlates of Overall Survival in Lower Grade Gliomas Using a Bag-of-words Paradigm: A Preliminary Analysis Based on Hematoxylin & Eosin Stained Slides from the Lower Grade Glioma Cohort of The Cancer Genome Atlas.
Reid Trenton Powell, Adriana Olar, +4 authors, Arvind Rao.
J Pathol Inform, 2017 Apr 07; 8. PMID: 28382223    Free PMC article.
A preliminary examination of the diagnostic value of deep learning in hip osteoarthritis.
Yanping Xue, Rongguo Zhang, +2 authors, Tao Jiang.
PLoS One, 2017 Jun 03; 12(6). PMID: 28575070    Free PMC article.
Deep Learning for Classification of Colorectal Polyps on Whole-slide Images.
Bruno Korbar, Andrea M Olofson, +5 authors, Saeed Hassanpour.
J Pathol Inform, 2017 Aug 23; 8. PMID: 28828201    Free PMC article.
Automated Classification of Lung Cancer Types from Cytological Images Using Deep Convolutional Neural Networks.
Atsushi Teramoto, Tetsuya Tsukamoto, Yuka Kiriyama, Hiroshi Fujita.
Biomed Res Int, 2017 Sep 09; 2017. PMID: 28884120    Free PMC article.
ECG data compression using a neural network model based on multi-objective optimization.
Bo Zhang, Jiasheng Zhao, Xiao Chen, Jianhuang Wu.
PLoS One, 2017 Oct 04; 12(10). PMID: 28972986    Free PMC article.
A novel machine learning approach reveals latent vascular phenotypes predictive of renal cancer outcome.
Nathan Ing, Fangjin Huang, +9 authors, Beatrice S Knudsen.
Sci Rep, 2017 Oct 19; 7(1). PMID: 29038551    Free PMC article.
Rapid Intraoperative Diagnosis of Pediatric Brain Tumors Using Stimulated Raman Histology.
Todd C Hollon, Spencer Lewis, +10 authors, Daniel A Orringer.
Cancer Res, 2017 Nov 03; 78(1). PMID: 29093006    Free PMC article.
Induction of apoptosis in human glioma cell lines of various grades through the ROS-mediated mitochondrial pathway and caspase activation by Rhaponticum carthamoides transformed root extract.
Ewa Skała, Tomasz Kowalczyk, +7 authors, Przemysław Sitarek.
Mol Cell Biochem, 2017 Dec 15; 445(1-2). PMID: 29238899
Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks.
James M Brown, J Peter Campbell, +9 authors, Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium.
JAMA Ophthalmol, 2018 May 26; 136(7). PMID: 29801159    Free PMC article.
Highly Cited.
Deep Learning Solutions for Classifying Patients on Opioid Use.
Zhengping Che, Jennifer St Sauver, Hongfang Liu, Yan Liu.
AMIA Annu Symp Proc, 2018 Jun 02; 2017. PMID: 29854117    Free PMC article.
Information-Based Medicine in Glioma Patients: A Clinical Perspective.
Joeky Tamba Senders, Maya Harary, +5 authors, Omar Arnaout.
Comput Math Methods Med, 2018 Jul 17; 2018. PMID: 30008798    Free PMC article.
Development and validation of an endoscopic images-based deep learning model for detection with nasopharyngeal malignancies.
Chaofeng Li, Bingzhong Jing, +34 authors, Xing Lv.
Cancer Commun (Lond), 2018 Sep 27; 38(1). PMID: 30253801    Free PMC article.
Classification of the glioma grading using radiomics analysis.
Hwan-Ho Cho, Seung-Hak Lee, Jonghoon Kim, Hyunjin Park.
PeerJ, 2018 Dec 01; 6. PMID: 30498643    Free PMC article.
Deep convolutional neural network VGG-16 model for differential diagnosing of papillary thyroid carcinomas in cytological images: a pilot study.
Qing Guan, Yunjun Wang, +6 authors, Jun Xiang.
J Cancer, 2019 Oct 11; 10(20). PMID: 31598159    Free PMC article.
Digital Assessment of Stained Breast Tissue Images for Comprehensive Tumor and Microenvironment Analysis.
Shachi Mittal, Catalin Stoean, Andre Kajdacsy-Balla, Rohit Bhargava.
Front Bioeng Biotechnol, 2019 Nov 05; 7. PMID: 31681737    Free PMC article.
Comparing Artificial Intelligence Platforms for Histopathologic Cancer Diagnosis.
Andrew A Borkowski, Catherine P Wilson, +4 authors, Stephen M Mastorides.
Fed Pract, 2019 Nov 27; 36(10). PMID: 31768096    Free PMC article.
Automated Gleason Scoring and Tumor Quantification in Prostate Core Needle Biopsy Images Using Deep Neural Networks and Its Comparison with Pathologist-Based Assessment.
Han Suk Ryu, Min-Sun Jin, +10 authors, Hyeyoon Chang.
Cancers (Basel), 2019 Nov 27; 11(12). PMID: 31769420    Free PMC article.
An End-to-end System for Automatic Characterization of Iba1 Immunopositive Microglia in Whole Slide Imaging.
Alexander D Kyriazis, Shahriar Noroozizadeh, +9 authors, Guy Nir.
Neuroinformatics, 2018 Nov 09; 17(3). PMID: 30406865
Deep Learning Global Glomerulosclerosis in Transplant Kidney Frozen Sections.
Jon N Marsh, Matthew K Matlock, +4 authors, S Joshua Swamidass.
IEEE Trans Med Imaging, 2018 Jul 12; 37(12). PMID: 29994669    Free PMC article.
Viable and necrotic tumor assessment from whole slide images of osteosarcoma using machine-learning and deep-learning models.
Harish Babu Arunachalam, Rashika Mishra, +6 authors, Patrick Leavey.
PLoS One, 2019 Apr 18; 14(4). PMID: 30995247    Free PMC article.
Multi-Path Dilated Residual Network for Nuclei Segmentation and Detection.
Eric Ke Wang, Xun Zhang, +4 authors, Nie Zhe.
Cells, 2019 May 28; 8(5). PMID: 31126166    Free PMC article.
Rapid histology of laryngeal squamous cell carcinoma with deep-learning based stimulated Raman scattering microscopy.
Lili Zhang, Yongzheng Wu, +11 authors, Minbiao Ji.
Theranostics, 2019 May 28; 9(9). PMID: 31131052    Free PMC article.
Deep learning techniques for detecting preneoplastic and neoplastic lesions in human colorectal histological images.
Paola Sena, Rita Fioresi, +3 authors, Luca Roncucci.
Oncol Lett, 2019 Dec 04; 18(6). PMID: 31788084    Free PMC article.
Emerging role of deep learning-based artificial intelligence in tumor pathology.
Yahui Jiang, Meng Yang, +2 authors, Yan Sun.
Cancer Commun (Lond), 2020 Apr 12; 40(4). PMID: 32277744    Free PMC article.
Deep learning assessment of breast terminal duct lobular unit involution: Towards automated prediction of breast cancer risk.
Suzanne C Wetstein, Allison M Onken, +13 authors, Mitko Veta.
PLoS One, 2020 Apr 16; 15(4). PMID: 32294107    Free PMC article.
Isocitrate dehydrogenase (IDH) status prediction in histopathology images of gliomas using deep learning.
Sidong Liu, Zubair Shah, +5 authors, Antonio Di Ieva.
Sci Rep, 2020 May 10; 10(1). PMID: 32382048    Free PMC article.
Predicting ischemic stroke tissue fate using a deep convolutional neural network on source magnetic resonance perfusion images.
King Chung Ho, Fabien Scalzo, +3 authors, Corey Arnold.
J Med Imaging (Bellingham), 2019 May 28; 6(2). PMID: 31131293    Free PMC article.
Measurement-oriented deep-learning workflow for improved segmentation of myelin and axons in high-resolution images of human cerebral white matter.
Predrag Janjic, Kristijan Petrovski, +11 authors, Andrew J Dwork.
J Neurosci Methods, 2019 Aug 05; 326. PMID: 31377177    Free PMC article.
Learning to Evaluate Color Similarity for Histopathology Images using Triplet Networks.
Anirudh Choudhary, Hang Wu, Li Tong, May D Wang.
ACM BCB, 2020 Jun 20; 2019. PMID: 32558828    Free PMC article.
Machine learning assisted DSC-MRI radiomics as a tool for glioma classification by grade and mutation status.
Carole H Sudre, Jasmina Panovska-Griffiths, +26 authors, Sotirios Bisdas.
BMC Med Inform Decis Mak, 2020 Jul 08; 20(1). PMID: 32631306    Free PMC article.
Artificial intelligence-based morphological fingerprinting of megakaryocytes: a new tool for assessing disease in MPN patients.
Korsuk Sirinukunwattana, Alan Aberdeen, +7 authors, Daniel Royston.
Blood Adv, 2020 Jul 25; 4(14). PMID: 32706893    Free PMC article.
Automated identification of retinopathy of prematurity by image-based deep learning.
Yan Tong, Wei Lu, +2 authors, Yin Shen.
Eye Vis (Lond), 2020 Aug 09; 7. PMID: 32766357    Free PMC article.
Current Trends of Artificial Intelligence for Colorectal Cancer Pathology Image Analysis: A Systematic Review.
Nishant Thakur, Hongjun Yoon, Yosep Chong.
Cancers (Basel), 2020 Jul 17; 12(7). PMID: 32668721    Free PMC article.
Digital pathology and computational image analysis in nephropathology.
Laura Barisoni, Kyle J Lafata, +2 authors, Ulysses G J Balis.
Nat Rev Nephrol, 2020 Aug 28; 16(11). PMID: 32848206    Free PMC article.
Research advances in forensic diatom testing.
Yuanyuan Zhou, Yongjie Cao, +6 authors, Ping Huang.
Forensic Sci Res, 2020 Sep 18; 5(2). PMID: 32939425    Free PMC article.
Artificial Intelligence in Pathology.
Hye Yoon Chang, Chan Kwon Jung, +4 authors, Tae-Yeong Kwak.
J Pathol Transl Med, 2019 Jan 03; 53(1). PMID: 30599506    Free PMC article.
Highly Cited.
The Use of Screencasts with Embedded Whole-Slide Scans and Hyperlinks to Teach Anatomic Pathology in a Supervised Digital Environment.
Mary Wong, Joseph Frye, Stacey Kim, Alberto M Marchevsky.
J Pathol Inform, 2019 Jan 05; 9. PMID: 30607306    Free PMC article.
Machine Learning Models for Multiparametric Glioma Grading With Quantitative Result Interpretations.
Xiuying Wang, Dingqian Wang, +7 authors, Yingchao Liu.
Front Neurosci, 2019 Jan 29; 12. PMID: 30686996    Free PMC article.
High-throughput, high-resolution deep learning microscopy based on registration-free generative adversarial network.
Hao Zhang, Chunyu Fang, +4 authors, Peng Fei.
Biomed Opt Express, 2019 Mar 21; 10(3). PMID: 30891329    Free PMC article.
Segmentation of Glomeruli Within Trichrome Images Using Deep Learning.
Shruti Kannan, Laura A Morgan, +9 authors, Vijaya B Kolachalama.
Kidney Int Rep, 2019 Jul 19; 4(7). PMID: 31317118    Free PMC article.
Survival outcome prediction in cervical cancer: Cox models vs deep-learning model.
Koji Matsuo, Sanjay Purushotham, +4 authors, Lynda D Roman.
Am J Obstet Gynecol, 2018 Dec 26; 220(4). PMID: 30582927    Free PMC article.
Optimization of deep learning methods for visualization of tumor heterogeneity and brain tumor grading through digital pathology.
An Hoai Truong, Viktoriia Sharmanska, Clara Limbӓck-Stanic, Matthew Grech-Sollars.
Neurooncol Adv, 2020 Nov 17; 2(1). PMID: 33196039    Free PMC article.
Deep neural network models for computational histopathology: A survey.
Chetan L Srinidhi, Ozan Ciga, Anne L Martel.
Med Image Anal, 2020 Oct 14; 67. PMID: 33049577    Free PMC article.
Implementation of convolutional neural network approach for COVID-19 disease detection.
Emrah Irmak.
Physiol Genomics, 2020 Oct 24; 52(12). PMID: 33094700    Free PMC article.
Artificial intelligence neuropathologist for glioma classification using deep learning on hematoxylin and eosin stained slide images and molecular markers.
Lei Jin, Feng Shi, +10 authors, Jinsong Wu.
Neuro Oncol, 2020 Jul 15; 23(1). PMID: 32663285    Free PMC article.
AI Neuropathologist: an innovative technology enabling a faultless pathological diagnosis?
Takashi Komori.
Neuro Oncol, 2020 Oct 16; 23(1). PMID: 33059363    Free PMC article.
Automated Classification and Segmentation in Colorectal Images Based on Self-Paced Transfer Network.
Yao Yao, Shuiping Gou, +2 authors, Shuixiang He.
Biomed Res Int, 2021 Feb 06; 2021. PMID: 33542924    Free PMC article.
Improved Glioma Grading Using Deep Convolutional Neural Networks.
S Gutta, J Acharya, +2 authors, K S Nayak.
AJNR Am J Neuroradiol, 2020 Dec 12; 42(2). PMID: 33303522    Free PMC article.
Identification and Validation of Efficacy of Immunological Therapy for Lung Cancer From Histopathological Images Based on Deep Learning.
Yachao Yang, Jialiang Yang, +4 authors, Kaimei Huang.
Front Genet, 2021 Feb 27; 12. PMID: 33633793    Free PMC article.
Deep learning-based grading of ductal carcinoma in situ in breast histopathology images.
Suzanne C Wetstein, Nikolas Stathonikos, +5 authors, Mitko Veta.
Lab Invest, 2021 Feb 21; 101(4). PMID: 33608619    Free PMC article.
Classification of Diffuse Glioma Subtype from Clinical-Grade Pathological Images Using Deep Transfer Learning.
Sanghyuk Im, Jonghwan Hyeon, +4 authors, Tae-Jung Kim.
Sensors (Basel), 2021 Jun 03; 21(10). PMID: 34067934    Free PMC article.
Deep Neural Network Analysis of Pathology Images With Integrated Molecular Data for Enhanced Glioma Classification and Grading.
Linmin Pei, Karra A Jones, +2 authors, Khan M Iftekharuddin.
Front Oncol, 2021 Jul 20; 11. PMID: 34277415    Free PMC article.
Machine learning analysis of TCGA cancer data.
Jose Liñares-Blanco, Alejandro Pazos, Carlos Fernandez-Lozano.
PeerJ Comput Sci, 2021 Jul 30; 7. PMID: 34322589    Free PMC article.