Journal Article
. 2011 Feb; 135(2):233-42.
doi: 10.1043/1543-2165-135.2.233.

Observer variability in the interpretation of HER2/neu immunohistochemical expression with unaided and computer-aided digital microscopy

Marios A Gavrielides 1 Brandon D Gallas  Petra Lenz  Aldo Badano  Stephen M Hewitt  
  • PMID: 21284444
  •     36 References
  •     37 citations


Context: Observer variability in digital microscopy and the effect of computer-aided digital microscopy are underexamined areas in need of further research, considering the increasing use and future role of digital imaging in pathology. A reduction in observer variability using computer aids could enhance the statistical power of studies designed to determine the utility of new biomarkers and accelerate their incorporation in clinical practice.

Objectives: To quantify interobserver and intraobserver variability in immunohistochemical analysis of HER2/neu with digital microscopy and computer-aided digital microscopy, and to test the hypothesis that observer agreement in the quantitative assessment of HER2/neu immunohistochemical expression is increased with the use of computer-aided microscopy.

Design: A set of 335 digital microscopy images extracted from 64 breast cancer tissue slides stained with a HER2 antibody, were read by 14 observers in 2 reading modes: the unaided mode and the computer-aided mode. In the unaided mode, HER2 images were displayed on a calibrated color monitor with no other information, whereas in the computer-aided mode, observers were shown a HER2 image along with a corresponding feature plot showing computer-extracted values of membrane staining intensity and membrane completeness for the particular image under examination and, at the same time, mean feature values of the different HER2 categories. In both modes, observers were asked to provide a continuous score of HER2 expression.

Results: Agreement analysis performed on the output of the study showed significant improvement in both interobserver and intraobserver agreement when the computer-aided reading mode was used to evaluate preselected image fields.

Conclusion: The role of computer-aided digital microscopy in reducing observer variability in immunohistochemistry is promising.

Interpretation and quantification of immunostains.
T Seidal, A J Balaton, H Battifora.
Am J Surg Pathol, 2001 Nov 02; 25(9). PMID: 11688582
Automated quantitative analysis of tissue microarrays reveals an association between high Bcl-2 expression and improved outcome in melanoma.
Kyle A Divito, Aaron J Berger, +3 authors, Harriet M Kluger.
Cancer Res, 2004 Dec 03; 64(23). PMID: 15574790
Immunohistochemistry and quantitative analysis of protein expression.
Melissa Cregger, Aaron J Berger, David L Rimm.
Arch Pathol Lab Med, 2006 Jul 13; 130(7). PMID: 16831029
HER-2/neu in breast cancer: interobserver variability and performance of immunohistochemistry with 4 antibodies compared with fluorescent in situ hybridization.
T A Thomson, M M Hayes, +5 authors, R L Parker.
Mod Pathol, 2001 Nov 14; 14(11). PMID: 11706067
Intraclass correlations: uses in assessing rater reliability.
P E Shrout, J L Fleiss.
Psychol Bull, 1979 Mar 01; 86(2). PMID: 18839484
Highly Cited. Review.
The use of digital images in pathology.
P N Furness.
J Pathol, 1998 Jan 10; 183(3). PMID: 9422979
Tissue microarrays: bridging the gap between research and the clinic.
Till Braunschweig, Joon-Yong Chung, Stephen M Hewitt.
Expert Rev Proteomics, 2005 Jul 08; 2(3). PMID: 16000080
American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer.
Antonio C Wolff, M Elizabeth H Hammond, +22 authors, College of American Pathologists.
J Clin Oncol, 2006 Dec 13; 25(1). PMID: 17159189
Highly Cited.
Quantitative immunohistochemical evaluation of HER2/neu expression with HercepTestTM in breast carcinoma by image analysis.
Y Hatanaka, K Hashizume, +4 authors, Y Tani.
Pathol Int, 2001 Jan 10; 51(1). PMID: 11148461
Quantitative evaluation of HER-2/neu status in breast cancer by fluorescence in situ hybridization and by immunohistochemistry with image analysis.
H A Lehr, T W Jacobs, +2 authors, A M Gown.
Am J Clin Pathol, 2001 Jun 08; 115(6). PMID: 11392876
Satisfaction of search in diagnostic radiology.
K S Berbaum, E A Franken, +7 authors, G Y el-Khoury.
Invest Radiol, 1990 Feb 01; 25(2). PMID: 2312249
Comparing subjective and digital image analysis HER2/neu expression scores with conventional and modified FISH scores in breast cancer.
I Skaland, I Øvestad, +4 authors, J P A Baak.
J Clin Pathol, 2007 Apr 07; 61(1). PMID: 17412872
Enhanced accuracy and reliability of HER-2/neu immunohistochemical scoring using digital microscopy.
Kenneth Bloom, Douglas Harrington.
Am J Clin Pathol, 2004 May 21; 121(5). PMID: 15151201
Perspectives in tissue microarrays.
Till Braunschweig, Joon-Yong Chung, Stephen M Hewitt.
Comb Chem High Throughput Screen, 2004 Sep 24; 7(6). PMID: 15379629
Quantitative immunohistochemistry by measuring cumulative signal strength using commercially available software photoshop and matlab.
K A Matkowskyj, D Schonfeld, R V Benya.
J Histochem Cytochem, 2000 Jan 20; 48(2). PMID: 10639497
Automated quantitative assessment of HER-2/neu immunohistochemical expression in breast cancer.
Hela Masmoudi, Stephen M Hewitt, +2 authors, Marios A Gavrielides.
IEEE Trans Med Imaging, 2009 Jan 24; 28(6). PMID: 19164073    Free PMC article.
Digital microscopy imaging and new approaches in toxicologic pathology.
Bruce McCullough, Xiaoyou Ying, Thomas Monticello, Marc Bonnefoi.
Toxicol Pathol, 2004 Oct 27; 32 Suppl 2. PMID: 15503664
Semi-automated imaging system to quantitate Her-2/neu membrane receptor immunoreactivity in human breast cancer.
Aparna S Joshi, Girish M Sharangpani, +6 authors, Sanford H Barsky.
Cytometry A, 2007 Feb 27; 71(5). PMID: 17323351
Assessment of HER-2/neu status in breast cancer. Automated Cellular Imaging System (ACIS)-assisted quantitation of immunohistochemical assay achieves high accuracy in comparison with fluorescence in situ hybridization assay as the standard.
S Wang, M H Saboorian, +6 authors, R Ashfaq.
Am J Clin Pathol, 2001 Oct 17; 116(4). PMID: 11601134
Quantitative analysis of breast cancer tissue microarrays shows that both high and normal levels of HER2 expression are associated with poor outcome.
Robert L Camp, Marisa Dolled-Filhart, Bonnie L King, David L Rimm.
Cancer Res, 2003 Apr 03; 63(7). PMID: 12670887
Interobserver reproducibility of immunocytochemical estrogen- and progesterone receptor status assessment in breast cancer.
S Biesterfeld, U Veuskens, +2 authors, A Böcking.
Anticancer Res, 1996 Sep 01; 16(5A). PMID: 8917341
Inter-laboratory and inter-observer reproducibility of immunohistochemical assessment of the Ki-67 labelling index in a large multi-centre trial.
Michael Mengel, Reinhard von Wasielewski, +3 authors, Hans Kreipe.
J Pathol, 2002 Oct 11; 198(3). PMID: 12375261
Digital imaging in pathology: the case for standardization.
Yukako Yagi, John R Gilbertson.
J Telemed Telecare, 2005 May 20; 11(3). PMID: 15901437
Virtual microscopy.
M Lundin, J Lundin, J Isola.
J Clin Pathol, 2004 Nov 26; 57(12). PMID: 15563661    Free PMC article.
High inter-observer agreement in immunohistochemical evaluation of HER-2/neu expression in breast cancer: a multicentre GEFPICS study.
Magali Lacroix-Triki, Simone Mathoulin-Pelissier, +9 authors, Laurent Arnould.
Eur J Cancer, 2006 Sep 23; 42(17). PMID: 16989997
Virtual microscopy in prostate histopathology: simultaneous viewing of biopsies stained sequentially with hematoxylin and eosin, and alpha-methylacyl-coenzyme A racemase/p63 immunohistochemistry.
Henrik O Helin, Mikael E Lundin, +3 authors, Jorma Isola.
J Urol, 2006 Jan 13; 175(2). PMID: 16406979
Objective measurement of breast cancer oestrogen receptor status through digital image analysis.
R Mofidi, R Walsh, +6 authors, N O'Higgins.
Eur J Surg Oncol, 2003 Feb 01; 29(1). PMID: 12559071
Innovations in medical imaging and virtual microscopy.
Ronald S Weinstein.
Hum Pathol, 2005 May 14; 36(4). PMID: 15891989
Computerized image analysis of p53 and proliferating cell nuclear antigen expression in benign, hyperplastic, and malignant endometrium.
A S Elhafey, J C Papadimitriou, +3 authors, S G Silverberg.
Arch Pathol Lab Med, 2001 Jun 23; 125(7). PMID: 11419970
Evaluation of HER-2/neu immunohistochemical assay sensitivity and scoring on formalin-fixed and paraffin-processed cell lines and breast tumors: a comparative study involving results from laboratories in 21 countries.
Anthony Rhodes, Bharat Jasani, +2 authors, André J Balaton.
Am J Clin Pathol, 2002 Sep 11; 118(3). PMID: 12219783
Interobserver reproducibility of Her-2/neu protein overexpression in invasive breast carcinoma using the DAKO HercepTest.
Chih-Yi Hsu, Donald Ming-Tak Ho, +3 authors, Hung Chiang.
Am J Clin Pathol, 2002 Nov 14; 118(5). PMID: 12428788
A scoring system for immunohistochemical staining: consensus report of the task force for basic research of the EORTC-GCCG. European Organization for Research and Treatment of Cancer-Gynaecological Cancer Cooperative Group.
P J van Diest, P van Dam, +4 authors, I Vergote.
J Clin Pathol, 1998 Feb 14; 50(10). PMID: 9462258    Free PMC article.
HER-2 status in breast cancer: correlation of gene amplification by FISH with immunohistochemistry expression using advanced cellular imaging system.
Armando Ciampa, Bo Xu, +5 authors, Ashraf Khan.
Appl Immunohistochem Mol Morphol, 2006 Jun 21; 14(2). PMID: 16785779
Digital slide and virtual microscopy based routine and telepathology evaluation of routine gastrointestinal biopsy specimens.
B Molnar, L Berczi, +4 authors, Z Tulassay.
J Clin Pathol, 2003 Jun 05; 56(6). PMID: 12783970    Free PMC article.
Computer-assisted assessment of the human epidermal growth factor receptor 2 immunohistochemical assay in imaged histologic sections using a membrane isolation algorithm and quantitative analysis of positive controls.
Bonnie H Hall, Monica Ianosi-Irimie, +3 authors, David J Foran.
BMC Med Imaging, 2008 Jun 07; 8. PMID: 18534031    Free PMC article.
Automated subcellular localization and quantification of protein expression in tissue microarrays.
Robert L Camp, Gina G Chung, David L Rimm.
Nat Med, 2002 Oct 22; 8(11). PMID: 12389040
Highly Cited.
Membrane connectivity estimated by digital image analysis of HER2 immunohistochemistry is concordant with visual scoring and fluorescence in situ hybridization results: algorithm evaluation on breast cancer tissue microarrays.
Aida Laurinaviciene, Darius Dasevicius, +3 authors, Arvydas Laurinavicius.
Diagn Pathol, 2011 Sep 29; 6. PMID: 21943197    Free PMC article.
Quantitative comparison of immunohistochemical staining measured by digital image analysis versus pathologist visual scoring.
Anthony E Rizzardi, Arthur T Johnson, +5 authors, Stephen C Schmechel.
Diagn Pathol, 2012 Apr 21; 7. PMID: 22515559    Free PMC article.
Highly Cited.
Quantitation of HER2/neu expression in primary gastroesophageal adenocarcinomas using conventional light microscopy and quantitative image analysis.
Jennifer Jeung, Roshan Patel, +2 authors, Chen Liu.
Arch Pathol Lab Med, 2012 Jun 01; 136(6). PMID: 22646266    Free PMC article.
Reproducibility in the automated quantitative assessment of HER2/neu for breast cancer.
Tyler Keay, Catherine M Conway, +3 authors, Marios A Gavrielides.
J Pathol Inform, 2013 Aug 24; 4. PMID: 23967384    Free PMC article.
IHC Profiler: an open source plugin for the quantitative evaluation and automated scoring of immunohistochemistry images of human tissue samples.
Frency Varghese, Amirali B Bukhari, Renu Malhotra, Abhijit De.
PLoS One, 2014 May 08; 9(5). PMID: 24802416    Free PMC article.
Highly Cited.
Development of automated quantification methodologies of immunohistochemical markers to determine patterns of immune response in breast cancer: a retrospective cohort study.
Carlos López, Cristina Callau, +12 authors, Marylène Lejeune.
BMJ Open, 2014 Aug 06; 4(8). PMID: 25091015    Free PMC article.
Diagnosis of inflammatory bowel disease: Potential role of molecular biometrics.
Amosy E M'Koma.
World J Gastrointest Surg, 2014 Nov 28; 6(11). PMID: 25429322    Free PMC article.
Observer performance in the use of digital and optical microscopy for the interpretation of tissue-based biomarkers.
Marios A Gavrielides, Catherine Conway, +2 authors, Stephen M Hewitt.
Anal Cell Pathol (Amst), 2014 Jan 01; 2014. PMID: 25763314    Free PMC article.
Evaluating whole slide imaging: A working group opportunity.
Darren Treanor, Brandon D Gallas, Marios A Gavrielides, Stephen M Hewitt.
J Pathol Inform, 2015 Mar 17; 6. PMID: 25774315    Free PMC article.
Computer Aided Semi-Automated Evaluation of HER2 Immunodetection--A Robust Solution for Supporting the Accuracy of Anti HER2 Therapy.
Tamás Micsik, Gábor Kiszler, +4 authors, Béla Molnár.
Pathol Oncol Res, 2015 Mar 20; 21(4). PMID: 25788005
Whole slide imaging for human epidermal growth factor receptor 2 immunohistochemistry interpretation: Accuracy, Precision, and reproducibility studies for digital manual and paired glass slide manual interpretation.
David C Wilbur, Elena F Brachtel, +3 authors, Savitra Krishnamurthy.
J Pathol Inform, 2015 Jun 26; 6. PMID: 26110090    Free PMC article.
Epidermal growth factor receptor immunohistochemistry: new opportunities in metastatic colorectal cancer.
Ryan A Hutchinson, Richard A Adams, +3 authors, Peter W Hamilton.
J Transl Med, 2015 Jul 08; 13. PMID: 26149458    Free PMC article.
Comparison of the manual, semiautomatic, and automatic selection and leveling of hot spots in whole slide images for Ki-67 quantification in meningiomas.
Zaneta Swiderska, Anna Korzynska, +6 authors, Bartlomiej Grala.
Anal Cell Pathol (Amst), 2015 Aug 05; 2015. PMID: 26240787    Free PMC article.
The Food and Drug Administration Office of Women's Health: Impact of Science on Regulatory Policy: An Update.
Merina Elahi, Noha Eshera, +12 authors, Marsha B Henderson.
J Womens Health (Larchmt), 2016 Feb 13; 25(3). PMID: 26871618    Free PMC article.
Digital image analysis outperforms manual biomarker assessment in breast cancer.
Gustav Stålhammar, Nelson Fuentes Martinez, +9 authors, Johan Hartman.
Mod Pathol, 2016 Feb 27; 29(4). PMID: 26916072
Quantitative comparison and reproducibility of pathologist scoring and digital image analysis of estrogen receptor β2 immunohistochemistry in prostate cancer.
Anthony E Rizzardi, Xiaotun Zhang, +8 authors, Stephen C Schmechel.
Diagn Pathol, 2016 Jul 13; 11(1). PMID: 27401406    Free PMC article.
A novel quantitative immunohistochemistry method for precise protein measurements directly in formalin-fixed, paraffin-embedded specimens: analytical performance measuring HER2.
Kristian Jensen, Rikke Krusenstjerna-Hafstrøm, +2 authors, Helene Derand.
Mod Pathol, 2016 Oct 22; 30(2). PMID: 27767098
Expression and prognostic impact of matrix metalloproteinase-2 (MMP-2) in astrocytomas.
Rahimsan K Ramachandran, Mia D Sørensen, +2 authors, Bjarne W Kristensen.
PLoS One, 2017 Feb 25; 12(2). PMID: 28234925    Free PMC article.
Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer.
Michel E Vandenberghe, Marietta L J Scott, +3 authors, Craig Barker.
Sci Rep, 2017 Apr 06; 7. PMID: 28378829    Free PMC article.
Digital pathology imaging as a novel platform for standardization and globalization of quantitative nephropathology.
Laura Barisoni, Charlotte Gimpel, +8 authors, Stephen M Hewitt.
Clin Kidney J, 2017 Jun 07; 10(2). PMID: 28584625    Free PMC article.
A Clinicopathological Study of Various Oral Cancer Diagnostic Techniques.
G Ulaganathan, K Thanvir Mohamed Niazi, +4 authors, A Banumathi.
J Pharm Bioallied Sci, 2017 Dec 30; 9(Suppl 1). PMID: 29284926    Free PMC article.
Deep Convolutional Neural Networks Enable Discrimination of Heterogeneous Digital Pathology Images.
Pegah Khosravi, Ehsan Kazemi, +2 authors, Iman Hajirasouliha.
EBioMedicine, 2018 Jan 03; 27. PMID: 29292031    Free PMC article.
Highly Cited.
Immunohistochemical HER2 Status Evaluation in Breast Cancer Pathology Samples: A Multicenter, Parallel-Design Concordance Study.
Tülay Canda, Ekrem Yavuz, +9 authors, Burçe Göktaş.
Eur J Breast Health, 2018 Aug 21; 14(3). PMID: 30123882    Free PMC article.
Computational measurement of tumor immune microenvironment in gastric adenocarcinomas.
Young Hwan Chang, You Jeong Heo, +3 authors, Kyoung-Mee Kim.
Sci Rep, 2018 Sep 19; 8(1). PMID: 30224753    Free PMC article.
Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer.
David F Steiner, Robert MacDonald, +6 authors, Martin C Stumpe.
Am J Surg Pathol, 2018 Oct 13; 42(12). PMID: 30312179    Free PMC article.
Highly Cited.
Quantitative Image Analysis of Human Epidermal Growth Factor Receptor 2 Immunohistochemistry for Breast Cancer: Guideline From the College of American Pathologists.
Marilyn M Bui, Michael W Riben, +11 authors, M Elizabeth Hammond.
Arch Pathol Lab Med, 2019 Jan 16; 143(10). PMID: 30645156    Free PMC article.
Digital Image Analysis of Heterogeneous Tuberculosis Pulmonary Pathology in Non-Clinical Animal Models using Deep Convolutional Neural Networks.
Bryce C Asay, Blake Blue Edwards, +10 authors, Anne J Lenaerts.
Sci Rep, 2020 Apr 10; 10(1). PMID: 32269234    Free PMC article.
Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group.
Mohamed Amgad, Elisabeth Specht Stovgaard, +66 authors, International Immuno-Oncology Biomarker Working Group.
NPJ Breast Cancer, 2020 May 16; 6. PMID: 32411818    Free PMC article.
Pitfalls in assessing stromal tumor infiltrating lymphocytes (sTILs) in breast cancer.
Zuzana Kos, Elvire Roblin, +91 authors, International Immuno-Oncology Biomarker Working Group.
NPJ Breast Cancer, 2020 May 16; 6. PMID: 32411819    Free PMC article.
An Image Analysis Solution For Quantification and Determination of Immunohistochemistry Staining Reproducibility.
Elizabeth A Chlipala, Christine M Bendzinski, +5 authors, Brad Bolon.
Appl Immunohistochem Mol Morphol, 2019 May 15; 28(6). PMID: 31082827    Free PMC article.
Immunohistochemical Study of Ezrin Expression in Colorectal Carcinoma: A Comparative Study between Objective Method and Digital Quantitative Assessment.
Essam E Ayad, Yousra O Kamal Eldin, +2 authors, Hesham A Elmeligy.
Asian Pac J Cancer Prev, 2020 Apr 27; 21(4). PMID: 32334457    Free PMC article.
Color standardization and optimization in whole slide imaging.
Takashi Inoue, Yukako Yagi.
Clin Diagn Pathol, 2020 Oct 23; 4(1). PMID: 33088926    Free PMC article.
A combined histo-score based on tumor differentiation and lymphocytic infiltrate is a robust prognostic marker for mobile tongue cancer.
Inger-Heidi Bjerkli, Elin Hadler-Olsen, +5 authors, Sonja E Steigen.
Virchows Arch, 2020 Jul 02; 477(6). PMID: 32607687    Free PMC article.
Comparing Deep Learning and Immunohistochemistry in Determining the Site of Origin for Well-Differentiated Neuroendocrine Tumors.
Jordan Redemann, Fred A Schultz, +4 authors, Joshua A Hanson.
J Pathol Inform, 2020 Dec 22; 11. PMID: 33343993    Free PMC article.

;. PMID: 33574288

;. PMID: 33574291
Selection of Representative Histologic Slides in Interobserver Reproducibility Studies: Insights from Expert Review for Ovarian Carcinoma Subtype Classification.
Marios A Gavrielides, Brigitte M Ronnett, +2 authors, Jeffrey D Seidman.
J Pathol Inform, 2021 May 21; 12. PMID: 34012719    Free PMC article.