Journal Article
. 2020 Jun;15(6).
doi: 10.1371/journal.pone.0233678.

Deep learning-based survival prediction for multiple cancer types using histopathology images

Ellery Wulczyn 1 David F Steiner 1 Zhaoyang Xu 1 Apaar Sadhwani 1 Hongwu Wang 1 Isabelle Flament-Auvigne 2 Craig H Mermel 1 Po-Hsuan Cameron Chen 1 Yun Liu 1 Martin C Stumpe 1 
Affiliations
  • PMID: 32555646
  •     25 References
  •     1 citations

Abstract

Providing prognostic information at the time of cancer diagnosis has important implications for treatment and monitoring. Although cancer staging, histopathological assessment, molecular features, and clinical variables can provide useful prognostic insights, improving risk stratification remains an active research area. We developed a deep learning system (DLS) to predict disease specific survival across 10 cancer types from The Cancer Genome Atlas (TCGA). We used a weakly-supervised approach without pixel-level annotations, and tested three different survival loss functions. The DLS was developed using 9,086 slides from 3,664 cases and evaluated using 3,009 slides from 1,216 cases. In multivariable Cox regression analysis of the combined cohort including all 10 cancers, the DLS was significantly associated with disease specific survival (hazard ratio of 1.58, 95% CI 1.28-1.70, p<0.0001) after adjusting for cancer type, stage, age, and sex. In a per-cancer adjusted subanalysis, the DLS remained a significant predictor of survival in 5 of 10 cancer types. Compared to a baseline model including stage, age, and sex, the c-index of the model demonstrated an absolute 3.7% improvement (95% CI 1.0-6.5) in the combined cohort. Additionally, our models stratified patients within individual cancer stages, particularly stage II (p = 0.025) and stage III (p<0.001). By developing and evaluating prognostic models across multiple cancer types, this work represents one of the most comprehensive studies exploring the direct prediction of clinical outcomes using deep learning and histopathology images. Our analysis demonstrates the potential for this approach to provide significant prognostic information in multiple cancer types, and even within specific pathologic stages. However, given the relatively small number of cases and observed clinical events for a deep learning task of this type, we observed wide confidence intervals for model performance, thus highlighting that future work will benefit from larger datasets assembled for the purposes for survival modeling.

Identifying survival associated morphological features of triple negative breast cancer using multiple datasets.
Chao Wang, Thierry Pécot, +3 authors, Kun Huang.
J Am Med Inform Assoc, 2013 Apr 16; 20(4). PMID: 23585272    Free PMC article.
The University of California, San Francisco Cancer of the Prostate Risk Assessment score: a straightforward and reliable preoperative predictor of disease recurrence after radical prostatectomy.
Matthew R Cooperberg, David J Pasta, +4 authors, Peter R Carroll.
J Urol, 2005 May 10; 173(6). PMID: 15879786    Free PMC article.
Highly Cited.
Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features.
Kun-Hsing Yu, Ce Zhang, +4 authors, Michael Snyder.
Nat Commun, 2016 Aug 17; 7. PMID: 27527408    Free PMC article.
Highly Cited.
Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer.
Babak Ehteshami Bejnordi, Mitko Veta, +67 authors, Rui Venâncio.
JAMA, 2017 Dec 14; 318(22). PMID: 29234806    Free PMC article.
Highly Cited.
Covariance analysis of censored survival data.
N Breslow.
Biometrics, 1974 Mar 01; 30(1). PMID: 4813387
Highly Cited.
Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
F E Harrell, K L Lee, D B Mark.
Stat Med, 1996 Feb 28; 15(4). PMID: 8668867
Highly Cited. Review.
Adjuvant Chemotherapy Guided by a 21-Gene Expression Assay in Breast Cancer.
Joseph A Sparano, Robert J Gray, +27 authors, George W Sledge.
N Engl J Med, 2018 Jun 05; 379(2). PMID: 29860917    Free PMC article.
Highly Cited.
Predicting cancer outcomes from histology and genomics using convolutional networks.
Pooya Mobadersany, Safoora Yousefi, +5 authors, Lee A D Cooper.
Proc Natl Acad Sci U S A, 2018 Mar 14; 115(13). PMID: 29531073    Free PMC article.
Highly Cited.
Genomic Classification of Cutaneous Melanoma.
Cancer Genome Atlas Network.
Cell, 2015 Jun 20; 161(7). PMID: 26091043    Free PMC article.
Highly Cited.
Deep learning based tissue analysis predicts outcome in colorectal cancer.
Dmitrii Bychkov, Nina Linder, +7 authors, Johan Lundin.
Sci Rep, 2018 Feb 23; 8(1). PMID: 29467373    Free PMC article.
Highly Cited.
Artificial Intelligence-Based Breast Cancer Nodal Metastasis Detection: Insights Into the Black Box for Pathologists.
Yun Liu, Timo Kohlberger, +7 authors, Martin C Stumpe.
Arch Pathol Lab Med, 2018 Oct 09; 143(7). PMID: 30295070
Highly Cited.
The Cancer Genome Atlas Pan-Cancer analysis project.
Cancer Genome Atlas Research Network, John N Weinstein, +7 authors, Joshua M Stuart.
Nat Genet, 2013 Sep 28; 45(10). PMID: 24071849    Free PMC article.
Highly Cited.
Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study.
Jakob Nikolas Kather, Johannes Krisam, +15 authors, Niels Halama.
PLoS Med, 2019 Jan 25; 16(1). PMID: 30677016    Free PMC article.
Highly Cited.
PanCancer insights from The Cancer Genome Atlas: the pathologist's perspective.
Lee Ad Cooper, Elizabeth G Demicco, +3 authors, Alexander J Lazar.
J Pathol, 2017 Dec 31; 244(5). PMID: 29288495    Free PMC article.
Review.
Prognostic significance of Nottingham histologic grade in invasive breast carcinoma.
Emad A Rakha, Maysa E El-Sayed, +5 authors, Ian O Ellis.
J Clin Oncol, 2008 May 21; 26(19). PMID: 18490649
Highly Cited.
Predicting survival after curative colectomy for cancer: individualizing colon cancer staging.
Martin R Weiser, Mithat Gönen, +2 authors, Deborah Schrag.
J Clin Oncol, 2011 Nov 16; 29(36). PMID: 22084366    Free PMC article.
Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs.
Varun Gulshan, Lily Peng, +12 authors, Dale R Webster.
JAMA, 2016 Nov 30; 316(22). PMID: 27898976
Highly Cited.
Deep learning.
Yann LeCun, Yoshua Bengio, Geoffrey Hinton.
Nature, 2015 May 29; 521(7553). PMID: 26017442
Highly Cited. Review.
Evaluating the yield of medical tests.
F E Harrell, R M Califf, +2 authors, R A Rosati.
JAMA, 1982 May 14; 247(18). PMID: 7069920
Highly Cited.
An Integrated TCGA Pan-Cancer Clinical Data Resource to Drive High-Quality Survival Outcome Analytics.
Jianfang Liu, Tara Lichtenberg, +13 authors, Hai Hu.
Cell, 2018 Apr 07; 173(2). PMID: 29625055    Free PMC article.
Highly Cited.
Association of Omics Features with Histopathology Patterns in Lung Adenocarcinoma.
Kun-Hsing Yu, Gerald J Berry, +3 authors, Michael Snyder.
Cell Syst, 2017 Nov 21; 5(6). PMID: 29153840    Free PMC article.
Clinical and Genomic Risk to Guide the Use of Adjuvant Therapy for Breast Cancer.
Joseph A Sparano, Robert J Gray, +27 authors, George W Sledge.
N Engl J Med, 2019 Jun 04; 380(25). PMID: 31157962    Free PMC article.
Highly Cited.
Systematic analysis of breast cancer morphology uncovers stromal features associated with survival.
Andrew H Beck, Ankur R Sangoi, +6 authors, Daphne Koller.
Sci Transl Med, 2011 Nov 11; 3(108). PMID: 22072638
Highly Cited.
Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer.
Kunal Nagpal, Davis Foote, +16 authors, Martin C Stumpe.
NPJ Digit Med, 2019 Jul 16; 2. PMID: 31304394    Free PMC article.
Highly Cited.
Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images.
Joel Saltz, Rajarsi Gupta, +14 authors, Vésteinn Thorsson.
Cell Rep, 2018 Apr 05; 23(1). PMID: 29617659    Free PMC article.
Highly Cited.
Development and Validation of a Deep Learning Algorithm for Gleason Grading of Prostate Cancer From Biopsy Specimens.
Kunal Nagpal, Davis Foote, +23 authors, Craig H Mermel.
JAMA Oncol, 2020 Jul 24; 6(9). PMID: 32701148    Free PMC article.