Journal Article
. 2009 Oct;10(11).
doi: 10.1016/S1470-2045(09)70254-2.

Calibration and discriminatory accuracy of prognosis calculation for breast cancer with the online Adjuvant! program: a hospital-based retrospective cohort study

Stella Mook 1 Marjanka K Schmidt  Emiel J Rutgers  Anthonie O van de Velde  Otto Visser  Sterre M Rutgers  Nicola Armstrong  Laura J van't Veer  Peter M Ravdin  
  • PMID: 19801202
  •     52 citations


Background: Adjuvant! is a web-based program that calculates individualised 10-year survival probabilities and predicted benefit of adjuvant systemic therapy. The Adjuvant! model has not been validated in any large European series. The aim of our study was to validate Adjuvant! in Dutch patients, investigating both its calibration and discriminatory accuracy.

Methods: Patients who were at least partly treated at the Netherlands Cancer Institute for breast cancer between 1987 and 1998 were included if they met the following criteria: tumour size T1 (< or =2 cm), T2 (2-5 cm), or T3 (>5 cm), invasive breast carcinoma, with information about involvement of axillary lymph nodes available, no distant metastases, primary surgery, axillary staging, and radiotherapy according to national guidelines. Clinicopathological characteristics and adjuvant treatment data were retrieved from hospital records and medical registries and were entered into the Adjuvant! (version 8.0) batch processor with blinding to outcome. Endpoints were overall survival and the proportion of patients that did not die from breast cancer (breast-cancer-specific survival [BCSS]).

Findings: 5380 patients were included with median follow-up of 11.7 years (range 0.03-21.8). The 10-year observed overall survival (69.0%) and BCSS (78.6%) and Adjuvant! predicted overall survival (69.1%) and BCSS (77.8%) were not statistically different (p=0.87 and p=0.18, respectively). Moreover, differences between predicted and observed outcomes were within 2% for most relevant clinicopathological subgroups. In patients younger than 40 years, Adjuvant! overestimated overall survival by 4.2% (p=0.04) and BCSS by 4.7% (p=0.01). The concordance index, which indicates discriminatory accuracy at the individual level, was 0.71 for BCSS in the entire cohort.

Interpretation: Adjuvant! accurately predicted 10-year outcomes in this large-scale Dutch validation study and is of use for adjuvant treatment decision making, although the results may be less reliable in some subgroups.

Tumor size and survival in breast cancer--a reappraisal.
William D Foulkes, Jorge S Reis-Filho, Steven A Narod.
Nat Rev Clin Oncol, 2010 Mar 24; 7(6). PMID: 20309006
Histological types of breast cancer: how special are they?
Britta Weigelt, Felipe C Geyer, Jorge S Reis-Filho.
Mol Oncol, 2010 May 11; 4(3). PMID: 20452298    Free PMC article.
Highly Cited. Review.
Breast cancer prognostic classification in the molecular era: the role of histological grade.
Emad A Rakha, Jorge S Reis-Filho, +15 authors, Ian O Ellis.
Breast Cancer Res, 2010 Sep 02; 12(4). PMID: 20804570    Free PMC article.
Highly Cited. Review.
Breast cancer assessment tools and optimizing adjuvant therapy.
Catherine Oakman, Libero Santarpia, Angelo Di Leo.
Nat Rev Clin Oncol, 2010 Oct 27; 7(12). PMID: 20975745
Derivation of cancer diagnostic and prognostic signatures from gene expression data.
Steve Goodison, Yijun Sun, Virginia Urquidi.
Bioanalysis, 2010 Nov 19; 2(5). PMID: 21083217    Free PMC article.
Molecular profiling currently offers no more than tumour morphology and basic immunohistochemistry.
Britta Weigelt, Jorge S Reis-Filho.
Breast Cancer Res, 2011 Jan 05; 12 Suppl 4. PMID: 21172089    Free PMC article.
Microarrays in the 2010s: the contribution of microarray-based gene expression profiling to breast cancer classification, prognostication and prediction.
Pierre-Emmanuel Colombo, Fernanda Milanezi, Britta Weigelt, Jorge S Reis-Filho.
Breast Cancer Res, 2011 Jul 27; 13(3). PMID: 21787441    Free PMC article.
Challenges translating breast cancer gene signatures into the clinic.
Britta Weigelt, Lajos Pusztai, Alan Ashworth, Jorge S Reis-Filho.
Nat Rev Clin Oncol, 2011 Sep 01; 9(1). PMID: 21878891
External validation of Adjuvant! Online breast cancer prognosis tool. Prioritising recommendations for improvement.
David Hajage, Yann de Rycke, +11 authors, Fabien Reyal.
PLoS One, 2011 Nov 17; 6(11). PMID: 22087319    Free PMC article.
PREDICT Plus: development and validation of a prognostic model for early breast cancer that includes HER2.
G C Wishart, C D Bajdik, +34 authors, P D P Pharoah.
Br J Cancer, 2012 Aug 02; 107(5). PMID: 22850554    Free PMC article.
A simple decision analytic solution to the comparison of two binary diagnostic tests.
Andrew J Vickers, Angel M Cronin, Mithat Gönen.
Stat Med, 2012 Sep 15; 32(11). PMID: 22975863    Free PMC article.
Accuracy validation of adjuvant! online in Taiwanese breast cancer patients--a 10-year analysis.
Kuo Yao-Lung, Chen Dar-Ren, Chang Tsai-Wang.
BMC Med Inform Decis Mak, 2012 Sep 19; 12. PMID: 22985190    Free PMC article.
A multichannel Markov random field framework for tumor segmentation with an application to classification of gene expression-based breast cancer recurrence risk.
Ahmed B Ashraf, Sara C Gavenonis, +3 authors, Despina Kontos.
IEEE Trans Med Imaging, 2012 Sep 26; 32(4). PMID: 23008246    Free PMC article.
Utility of prognostic genomic tests in breast cancer practice: The IMPAKT 2012 Working Group Consensus Statement.
H A Azim, S Michiels, +9 authors, C Swanton.
Ann Oncol, 2013 Jan 23; 24(3). PMID: 23337633    Free PMC article.
Risk assessment, disease prevention and personalised treatments in breast cancer: is clinically qualified integrative approach in the horizon?
Olga Golubnitschaja, Kristina Yeghiazaryan, +5 authors, Hans H Schild.
EPMA J, 2013 Feb 20; 4(1). PMID: 23418957    Free PMC article.
Gene expression profiling in breast cancer.
Belisario A Arango, Celine L Rivera, Stefan Glück.
Am J Transl Res, 2013 Apr 11; 5(2). PMID: 23573359    Free PMC article.
Underestimated survival predictions of the prognostic tools Adjuvant! Online and PREDICT in BRCA1-associated breast cancer patients.
Grigorijs Plakhins, Arvids Irmejs, +7 authors, Janis Gardovskis.
Fam Cancer, 2013 May 04; 12(4). PMID: 23640109
Progesterone receptor expression is an independent prognostic variable in early breast cancer: a population-based study.
C A Purdie, P Quinlan, +4 authors, A M Thompson.
Br J Cancer, 2013 Dec 05; 110(3). PMID: 24300977    Free PMC article.
Agreement on indication for systemic therapy between biopsied tissue and surgical excision specimens in breast cancer patients.
Alexander M Th Schmitz, Joost J Oudejans, Kenneth G A Gilhuijs.
PLoS One, 2014 Mar 13; 9(3). PMID: 24614128    Free PMC article.
Modern Risk Assessment for Individualizing Treatment Concepts in Early-stage Breast Cancer.
Alex Farr, Rachel Wuerstlein, +2 authors, Nadia Harbeck.
Rev Obstet Gynecol, 2013 Jan 01; 6(3-4). PMID: 24920978    Free PMC article.
Effect of PREDICT on chemotherapy/trastuzumab recommendations in HER2-positive patients with early-stage breast cancer.
Sue K Down, Olivia Lucas, John R Benson, Gordon C Wishart.
Oncol Lett, 2014 Nov 05; 8(6). PMID: 25364461    Free PMC article.
Are we able to predict survival in ER-positive HER2-negative breast cancer? A comparison of web-based models.
E Laas, P Mallon, +7 authors, F Reyal.
Br J Cancer, 2015 Jan 16; 112(5). PMID: 25590666    Free PMC article.
Comparison of Nottingham Prognostic Index and Adjuvant Online prognostic tools in young women with breast cancer: review of a single-institution experience.
Benjamin Joseph Hearne, M Dawn Teare, Mohammad Butt, Leslie Donaldson.
BMJ Open, 2015 Jan 30; 5(1). PMID: 25628047    Free PMC article.
The predictive accuracy of PREDICT: a personalized decision-making tool for Southeast Asian women with breast cancer.
Hoong-Seam Wong, Shridevi Subramaniam, +7 authors, Nirmala Bhoo-Pathy.
Medicine (Baltimore), 2015 Feb 26; 94(8). PMID: 25715267    Free PMC article.
Telomere fusion threshold identifies a poor prognostic subset of breast cancer patients.
K Simpson, R E Jones, +3 authors, D M Baird.
Mol Oncol, 2015 Mar 11; 9(6). PMID: 25752197    Free PMC article.
Prediction of breast cancer survival using clinical and genetic markers by tumor subtypes.
Nan Song, Ji-Yeob Choi, +12 authors, Daehee Kang.
PLoS One, 2015 Apr 14; 10(4). PMID: 25867717    Free PMC article.
Progress in adjuvant chemotherapy for breast cancer: an overview.
Jesus Anampa, Della Makower, Joseph A Sparano.
BMC Med, 2015 Aug 19; 13. PMID: 26278220    Free PMC article.
Highly Cited. Review.
The Performance of the Nottingham Prognosis Index and the Adjuvant Online Decision Making Tool for Prognosis in Early-stage Breast Cancer Patients.
Mehri Rejali, Mehdi Tazhibi, +2 authors, Reyhane Mokarian.
Int J Prev Med, 2015 Nov 26; 6. PMID: 26605014    Free PMC article.
Biomarkers in Breast Cancer - An Update.
M Schmidt, P A Fasching, M W Beckmann, H Kölbl.
Geburtshilfe Frauenheilkd, 2012 Sep 01; 72(9). PMID: 26640290    Free PMC article.
Validity of the online PREDICT tool in older patients with breast cancer: a population-based study.
N A de Glas, E Bastiaannet, +6 authors, J E A Portielje.
Br J Cancer, 2016 Jan 20; 114(4). PMID: 26783995    Free PMC article.
Breast cancer classification and prognostication through diverse systems along with recent emerging findings in this respect; the dawn of new perspectives in the clinical applications.
Vida Pourteimoor, Samira Mohammadi-Yeganeh, Mahdi Paryan.
Tumour Biol, 2016 Sep 22; 37(11). PMID: 27651157
Prognostic models in male breast cancer.
Carmen C van der Pol, Miangela M Lacle, +7 authors, Paul J van Diest.
Breast Cancer Res Treat, 2016 Sep 28; 160(2). PMID: 27671991    Free PMC article.
Comparison of the Ability of Different Clinical Treatment Scores to Estimate Prognosis in High-Risk Early Breast Cancer Patients: A Hellenic Cooperative Oncology Group Study.
Flora Stavridi, Konstantine T Kalogeras, +19 authors, George Fountzilas.
PLoS One, 2016 Oct 04; 11(10). PMID: 27695115    Free PMC article.
Validation of the CancerMath prognostic tool for breast cancer in Southeast Asia.
Hui Miao, Mikael Hartman, +8 authors, Nirmala Bhoo-Pathy.
BMC Cancer, 2016 Oct 23; 16(1). PMID: 27769212    Free PMC article.
The prognostic performance of Adjuvant! Online and Nottingham Prognostic Index in young breast cancer patients.
Matteo Lambertini, Ana Catarina Pinto, +12 authors, Hatem A Azim.
Br J Cancer, 2016 Nov 02; 115(12). PMID: 27802449    Free PMC article.
A Comprehensive Model for Predicting Recurrence and Survival in Cases of Chinese Postoperative Invasive Breast Cancer
Xianhe Xie, Yanfen Hu, +7 authors, Weili Zheng.
Asian Pac J Cancer Prev, 2017 Apr 27; 18(3). PMID: 28441706    Free PMC article.
Interaction of mammographic breast density with menopausal status and postmenopausal hormone use in relation to the risk of aggressive breast cancer subtypes.
Lusine Yaghjyan, Rulla M Tamimi, +15 authors, Celine M Vachon.
Breast Cancer Res Treat, 2017 Jun 19; 165(2). PMID: 28624977    Free PMC article.
Internet tools to enhance breast cancer care.
Shlomit Strulov Shachar, Hyman B Muss.
NPJ Breast Cancer, 2016 Apr 27; 2. PMID: 28721377    Free PMC article.
Early health technology assessment of magnetic resonance-guided high intensity focused ultrasound ablation for the treatment of early-stage breast cancer.
Floortje M Knuttel, Sèvrin E M Huijsse, +5 authors, Geertruida H de Bock.
J Ther Ultrasound, 2017 Aug 07; 5. PMID: 28781881    Free PMC article.
Are online prediction tools a valid alternative to genomic profiling in the context of systemic treatment of ER-positive breast cancer?
Umar Wazir, Kinan Mokbel, Amtul Carmichael, Kefah Mokbel.
Cell Mol Biol Lett, 2017 Sep 08; 22. PMID: 28878809    Free PMC article.
Preoperative indication for systemic therapy extended to patients with early-stage breast cancer using multiparametric 7-tesla breast MRI.
A M T Schmitz, W B Veldhuis, +8 authors, K G A Gilhuijs.
PLoS One, 2017 Sep 28; 12(9). PMID: 28949967    Free PMC article.
Long-term prognosis of young breast cancer patients (≤40 years) who did not receive adjuvant systemic treatment: protocol for the PARADIGM initiative cohort study.
Gwen Mhe Dackus, Natalie D Ter Hoeve, +30 authors, Sabine C Linn.
BMJ Open, 2017 Nov 16; 7(11). PMID: 29138205    Free PMC article.
Clinical decision making in cancer care: a review of current and future roles of patient age.
Eirik Joakim Tranvåg, Ole Frithjof Norheim, Trygve Ottersen.
BMC Cancer, 2018 May 11; 18(1). PMID: 29743048    Free PMC article.
Clinical implication of subcategorizing T2 category into T2a and T2b in TNM staging of breast cancer.
Jiwoong Jung, Young Jin Suh, +5 authors, Ki-Tae Hwang.
Cancer Med, 2018 Oct 13; 7(11). PMID: 30311421    Free PMC article.
A Nomogram To Predict The Overall Survival Of Breast Cancer Patients And Guide The Postoperative Adjuvant Chemotherapy In China.
Xin Wang, Ziwei Feng, +12 authors, Kexin Chen.
Cancer Manag Res, 2019 Dec 11; 11. PMID: 31819635    Free PMC article.
Decision of Adjuvant Systemic Treatment in HR+ HER2- Early Invasive Breast Cancer: Which Biomarkers Could Help?
Marie Alexandre, Aurélie Maran-Gonzalez, +6 authors, Séverine Guiu.
Cancer Manag Res, 2019 Dec 19; 11. PMID: 31849525    Free PMC article.
Clinical and Genomic Risk to Guide the Use of Adjuvant Therapy for Breast Cancer.
Joseph A Sparano, Robert J Gray, +27 authors, George W Sledge.
N Engl J Med, 2019 Jun 04; 380(25). PMID: 31157962    Free PMC article.
Highly Cited.
Facilitating validation of prediction models: a comparison of manual and semi-automated validation using registry-based data of breast cancer patients in the Netherlands.
Cornelia D van Steenbeek, Marissa C van Maaren, +3 authors, Hendrik Koffijberg.
BMC Med Res Methodol, 2019 Jun 10; 19(1). PMID: 31176362    Free PMC article.
Computer-Assisted Nuclear Atypia Scoring of Breast Cancer: a Preliminary Study.
Ziba Gandomkar, Patrick C Brennan, Claudia Mello-Thoms.
J Digit Imaging, 2019 Feb 06; 32(5). PMID: 30719586    Free PMC article.
A Comparison Between the Online Prediction Models CancerMath and PREDICT as Prognostic Tools in Thai Breast Cancer Patients.
Nuanphan Polchai, Doonyapat Sa-Nguanraksa, +3 authors, Pornchai O-Charoenrat.
Cancer Manag Res, 2020 Aug 06; 12. PMID: 32753968    Free PMC article.
Multidimensional Machine Learning Personalized Prognostic Model in an Early Invasive Breast Cancer Population-Based Cohort in China: Algorithm Validation Study.
Xiaorong Zhong, Ting Luo, +13 authors, Hong Zheng.
JMIR Med Inform, 2020 Nov 10; 8(11). PMID: 33164899    Free PMC article.
Histological Aspects and Quantitative Assessment of Ki67 as Prognostic Factors in Breast Cancer Patients: Result from a Single-Center, Cross Sectional Study.
Irina Niță, Cornelia Nițipir, +6 authors, Loredana Sabina Cornelia Manolescu.
Medicina (Kaunas), 2020 Nov 14; 56(11). PMID: 33182401    Free PMC article.