Journal Article
. 2019 Mar; 54:111-121.
doi: 10.1016/

Predicting breast tumor proliferation from whole-slide images: The TUPAC16 challenge

Mitko Veta 1 Yujing J Heng 2 Nikolas Stathonikos 3 Babak Ehteshami Bejnordi 4 Francisco Beca 5 Thomas Wollmann 6 Karl Rohr 6 Manan A Shah 7 Dayong Wang 2 Mikael Rousson 8 Martin Hedlund 8 David Tellez 5 Francesco Ciompi 5 Erwan Zerhouni 9 David Lanyi 9 Matheus Viana 10 Vassili Kovalev 11 Vitali Liauchuk 11 Hady Ahmady Phoulady 12 Talha Qaiser 13 Simon Graham 13 Nasir Rajpoot 13 Erik Sjöblom 14 Jesper Molin 14 Kyunghyun Paeng 15 Sangheum Hwang 15 Sunggyun Park 15 Zhipeng Jia 16 Eric I-Chao Chang 17 Yan Xu 18 Andrew H Beck 2 Paul J van Diest 3 Josien P W Pluim 19 
  • PMID: 30861443
  •     20 citations


Tumor proliferation is an important biomarker indicative of the prognosis of breast cancer patients. Assessment of tumor proliferation in a clinical setting is a highly subjective and labor-intensive task. Previous efforts to automate tumor proliferation assessment by image analysis only focused on mitosis detection in predefined tumor regions. However, in a real-world scenario, automatic mitosis detection should be performed in whole-slide images (WSIs) and an automatic method should be able to produce a tumor proliferation score given a WSI as input. To address this, we organized the TUmor Proliferation Assessment Challenge 2016 (TUPAC16) on prediction of tumor proliferation scores from WSIs. The challenge dataset consisted of 500 training and 321 testing breast cancer histopathology WSIs. In order to ensure fair and independent evaluation, only the ground truth for the training dataset was provided to the challenge participants. The first task of the challenge was to predict mitotic scores, i.e., to reproduce the manual method of assessing tumor proliferation by a pathologist. The second task was to predict the gene expression based PAM50 proliferation scores from the WSI. The best performing automatic method for the first task achieved a quadratic-weighted Cohen's kappa score of κ = 0.567, 95% CI [0.464, 0.671] between the predicted scores and the ground truth. For the second task, the predictions of the top method had a Spearman's correlation coefficient of r = 0.617, 95% CI [0.581 0.651] with the ground truth. This was the first comparison study that investigated tumor proliferation assessment from WSIs. The achieved results are promising given the difficulty of the tasks and weakly-labeled nature of the ground truth. However, further research is needed to improve the practical utility of image analysis methods for this task.

Keywords: Breast cancer; Cancer prognostication; Deep learning; Tumor proliferation.

Translational AI and Deep Learning in Diagnostic Pathology.
Ahmed Serag, Adrian Ion-Margineanu, +5 authors, Peter Hamilton.
Front Med (Lausanne), 2019 Oct 22; 6. PMID: 31632973    Free PMC article.
A large-scale dataset for mitotic figure assessment on whole slide images of canine cutaneous mast cell tumor.
Christof A Bertram, Marc Aubreville, +2 authors, Robert Klopfleisch.
Sci Data, 2019 Nov 23; 6(1). PMID: 31754105    Free PMC article.
Deep Learning for Whole Slide Image Analysis: An Overview.
Neofytos Dimitriou, Ognjen Arandjelović, Peter D Caie.
Front Med (Lausanne), 2019 Dec 12; 6. PMID: 31824952    Free PMC article.
Semi-Supervised Nests of Melanocytes Segmentation Method Using Convolutional Autoencoders.
Dariusz Kucharski, Pawel Kleczek, +2 authors, Marek Gorgon.
Sensors (Basel), 2020 Mar 15; 20(6). PMID: 32168748    Free PMC article.
Emerging role of deep learning-based artificial intelligence in tumor pathology.
Yahui Jiang, Meng Yang, +2 authors, Yan Sun.
Cancer Commun (Lond), 2020 Apr 12; 40(4). PMID: 32277744    Free PMC article.
Deep learning assessment of breast terminal duct lobular unit involution: Towards automated prediction of breast cancer risk.
Suzanne C Wetstein, Allison M Onken, +13 authors, Mitko Veta.
PLoS One, 2020 Apr 16; 15(4). PMID: 32294107    Free PMC article.
Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses.
Liron Pantanowitz, Douglas Hartman, +8 authors, Soo Youn Cho.
Diagn Pathol, 2020 Jul 06; 15(1). PMID: 32622359    Free PMC article.
Artificial intelligence in digital breast pathology: Techniques and applications.
Asmaa Ibrahim, Paul Gamble, +4 authors, Emad A Rakha.
Breast, 2020 Jan 15; 49. PMID: 31935669    Free PMC article.
Similar image search for histopathology: SMILY.
Narayan Hegde, Jason D Hipp, +11 authors, Martin C Stumpe.
NPJ Digit Med, 2019 Jul 16; 2. PMID: 31304402    Free PMC article.
Staining Invariant Features for Improving Generalization of Deep Convolutional Neural Networks in Computational Pathology.
Sebastian Otálora, Manfredo Atzori, +2 authors, Henning Müller.
Front Bioeng Biotechnol, 2019 Sep 12; 7. PMID: 31508414    Free PMC article.
Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region.
Marc Aubreville, Christof A Bertram, +9 authors, Andreas Maier.
Sci Rep, 2020 Oct 07; 10(1). PMID: 33020510    Free PMC article.
Assessment of tumour proliferation by use of the mitotic activity index, and Ki67 and phosphohistone H3 expression, in early-stage luminal breast cancer.
Julia E C van Steenhoven, Anne Kuijer, +4 authors, Paul J van Diest.
Histopathology, 2020 Jun 20; 77(4). PMID: 32557844    Free PMC article.
LibMI: An Open Source Library for Efficient Histopathological Image Processing.
Yuxin Dong, Pargorn Puttapirat, +2 authors, Chen Li.
J Pathol Inform, 2020 Oct 13; 11. PMID: 33042605    Free PMC article.
Interpretable multimodal deep learning for real-time pan-tissue pan-disease pathology search on social media.
Andrew J Schaumberg, Wendy C Juarez-Nicanor, +27 authors, Thomas J Fuchs.
Mod Pathol, 2020 May 30; 33(11). PMID: 32467650    Free PMC article.
Deep neural network models for computational histopathology: A survey.
Chetan L Srinidhi, Ozan Ciga, Anne L Martel.
Med Image Anal, 2020 Oct 14; 67. PMID: 33049577    Free PMC article.
A completely annotated whole slide image dataset of canine breast cancer to aid human breast cancer research.
Marc Aubreville, Christof A Bertram, +3 authors, Robert Klopfleisch.
Sci Data, 2020 Nov 29; 7(1). PMID: 33247116    Free PMC article.
Deep learning-based grading of ductal carcinoma in situ in breast histopathology images.
Suzanne C Wetstein, Nikolas Stathonikos, +5 authors, Mitko Veta.
Lab Invest, 2021 Feb 21; 101(4). PMID: 33608619    Free PMC article.
Deep Learning Image Analysis of Benign Breast Disease to Identify Subsequent Risk of Breast Cancer.
Adithya D Vellal, Korsuk Sirinukunwattan, +10 authors, Yujing J Heng.
JNCI Cancer Spectr, 2021 Mar 02; 5(1). PMID: 33644680    Free PMC article.
A multi-phase deep CNN based mitosis detection framework for breast cancer histopathological images.
Anabia Sohail, Asifullah Khan, +2 authors, Saranjam Khan.
Sci Rep, 2021 Mar 20; 11(1). PMID: 33737632    Free PMC article.
Deep learning in histopathology: the path to the clinic.
Jeroen van der Laak, Geert Litjens, Francesco Ciompi.
Nat Med, 2021 May 16; 27(5). PMID: 33990804