Journal Article
. 2014 Jan; 2014:157308.
doi: 10.1155/2014/157308.

Observer performance in the use of digital and optical microscopy for the interpretation of tissue-based biomarkers

Marios A Gavrielides 1 Catherine Conway 2 Neil O'Flaherty 1 Brandon D Gallas 1 Stephen M Hewitt 3 
  • PMID: 25763314
  •     50 References
  •     13 citations


Background: We conducted a validation study of digital pathology for the quantitative assessment of tissue-based biomarkers with immunohistochemistry.

Objective: To examine observer agreement as a function of viewing modality (digital versus optical microscopy), whole slide versus tissue microarray (TMA) review, biomarker type (HER2 incorporating membranous staining and Ki-67 with nuclear staining), and data type (continuous and categorical).

Methods: Eight pathologists reviewed 50 breast cancer whole slides (25 stained with HER2 and 25 with Ki-67) and 2 TMAs (1 stained with HER2, 1 with Ki-67, each containing 97 cores), using digital and optical microscopy.

Results: Results showed relatively high overall interobserver and intermodality agreement, with different patterns specific to biomarker type. For HER2, there was better interobserver agreement for optical compared to digital microscopy for whole slides as well as better interobserver and intermodality agreement for TMAs. For Ki-67, those patterns were not observed.

Conclusions: The differences in agreement patterns when examining different biomarkers and different scoring methods and reviewing whole slides compared to TMA stress the need for validation studies focused on specific pathology tasks to eliminate sources of variability that might dilute findings. The statistical uncertainty observed in our analyses calls for adequate sampling for each individual task rather than pooling cases.

Quantitative immunohistochemistry by measuring cumulative signal strength using commercially available software photoshop and matlab.
K A Matkowskyj, D Schonfeld, R V Benya.
J Histochem Cytochem, 2000 Jan 20; 48(2). PMID: 10639497
Quantitative immunohistochemical evaluation of HER2/neu expression with HercepTestTM in breast carcinoma by image analysis.
Y Hatanaka, K Hashizume, +4 authors, Y Tani.
Pathol Int, 2001 Jan 10; 51(1). PMID: 11148461
Quantitative evaluation of HER-2/neu status in breast cancer by fluorescence in situ hybridization and by immunohistochemistry with image analysis.
H A Lehr, T W Jacobs, +2 authors, A M Gown.
Am J Clin Pathol, 2001 Jun 08; 115(6). PMID: 11392876
Computerized image analysis of p53 and proliferating cell nuclear antigen expression in benign, hyperplastic, and malignant endometrium.
A S Elhafey, J C Papadimitriou, +3 authors, S G Silverberg.
Arch Pathol Lab Med, 2001 Jun 23; 125(7). PMID: 11419970
Assessment of HER-2/neu status in breast cancer. Automated Cellular Imaging System (ACIS)-assisted quantitation of immunohistochemical assay achieves high accuracy in comparison with fluorescence in situ hybridization assay as the standard.
S Wang, M H Saboorian, +6 authors, R Ashfaq.
Am J Clin Pathol, 2001 Oct 17; 116(4). PMID: 11601134
Objective measurement of breast cancer oestrogen receptor status through digital image analysis.
R Mofidi, R Walsh, +6 authors, N O'Higgins.
Eur J Surg Oncol, 2003 Feb 01; 29(1). PMID: 12559071
Digital slide and virtual microscopy based routine and telepathology evaluation of routine gastrointestinal biopsy specimens.
B Molnar, L Berczi, +4 authors, Z Tulassay.
J Clin Pathol, 2003 Jun 05; 56(6). PMID: 12783970    Free PMC article.
Development and evaluation of the virtual pathology slide: a new tool in telepathology.
Sean S P Costello, Daniel J Johnston, Peter A Dervan, Daniel G O'Shea.
J Med Internet Res, 2003 Jul 15; 5(2). PMID: 12857667    Free PMC article.
Guidelines for HER2 testing in the UK.
E D Hsi, R R Tubbs.
J Clin Pathol, 2004 Mar 03; 57(3). PMID: 14990591    Free PMC article.
Design, construction, and use of tissue microarrays.
Stephen M Hewitt.
Methods Mol Biol, 2004 Mar 17; 264. PMID: 15020780
Discordant results obtained for different methods of HER-2/neu testing in breast cancer--a question of standardization, automation and timing.
D Luftner, P Henschke, +7 authors, K Possinger.
Int J Biol Markers, 2004 Apr 14; 19(1). PMID: 15077921
Enhanced accuracy and reliability of HER-2/neu immunohistochemical scoring using digital microscopy.
Kenneth Bloom, Douglas Harrington.
Am J Clin Pathol, 2004 May 21; 121(5). PMID: 15151201
Digital microscopy imaging and new approaches in toxicologic pathology.
Bruce McCullough, Xiaoyou Ying, Thomas Monticello, Marc Bonnefoi.
Toxicol Pathol, 2004 Oct 27; 32 Suppl 2. PMID: 15503664
Automated quantitative analysis of tissue microarrays reveals an association between high Bcl-2 expression and improved outcome in melanoma.
Kyle A Divito, Aaron J Berger, +3 authors, Harriet M Kluger.
Cancer Res, 2004 Dec 03; 64(23). PMID: 15574790
Virtual microscopy in prostate histopathology: simultaneous viewing of biopsies stained sequentially with hematoxylin and eosin, and alpha-methylacyl-coenzyme A racemase/p63 immunohistochemistry.
Henrik O Helin, Mikael E Lundin, +3 authors, Jorma Isola.
J Urol, 2006 Jan 13; 175(2). PMID: 16406979
Comparison of immunohistochemistry by automated cellular imaging system (ACIS) versus fluorescence in-situ hybridization in the evaluation of HER-2/neu expression in primary breast carcinoma.
O W Tawfik, B F Kimler, +8 authors, C J Fabian.
Histopathology, 2006 Jan 25; 48(3). PMID: 16430472
Primary histologic diagnosis using automated whole slide imaging: a validation study.
John R Gilbertson, Jonhan Ho, +3 authors, Anil V Parwani.
BMC Clin Pathol, 2006 Apr 29; 6. PMID: 16643664    Free PMC article.
The development and validation of the Virtual Tissue Matrix, a software application that facilitates the review of tissue microarrays on line.
Catherine M Conway, Deirdre O'Shea, +9 authors, Daniel G O'Shea.
BMC Bioinformatics, 2006 May 19; 7. PMID: 16707006    Free PMC article.
HER-2 status in breast cancer: correlation of gene amplification by FISH with immunohistochemistry expression using advanced cellular imaging system.
Armando Ciampa, Bo Xu, +5 authors, Ashraf Khan.
Appl Immunohistochem Mol Morphol, 2006 Jun 21; 14(2). PMID: 16785779
Immunohistochemistry and quantitative analysis of protein expression.
Melissa Cregger, Aaron J Berger, David L Rimm.
Arch Pathol Lab Med, 2006 Jul 13; 130(7). PMID: 16831029
American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer.
Antonio C Wolff, M Elizabeth H Hammond, +22 authors, College of American Pathologists.
J Clin Oncol, 2006 Dec 13; 25(1). PMID: 17159189
Highly Cited.
Semi-automated imaging system to quantitate Her-2/neu membrane receptor immunoreactivity in human breast cancer.
Aparna S Joshi, Girish M Sharangpani, +6 authors, Sanford H Barsky.
Cytometry A, 2007 Feb 27; 71(5). PMID: 17323351
Comparing subjective and digital image analysis HER2/neu expression scores with conventional and modified FISH scores in breast cancer.
I Skaland, I Øvestad, +4 authors, J P A Baak.
J Clin Pathol, 2007 Apr 07; 61(1). PMID: 17412872
Evaluation of whole slide image immunohistochemistry interpretation in challenging prostate needle biopsies.
Jeffrey L Fine, Dana M Grzybicki, +8 authors, Drazen M Jukic.
Hum Pathol, 2008 Feb 01; 39(4). PMID: 18234276
Quantification of diverse subcellular immunohistochemical markers with clinicobiological relevancies: validation of a new computer-assisted image analysis procedure.
Marylène Lejeune, Joaquín Jaén, +8 authors, Tomás Alvaro.
J Anat, 2008 May 31; 212(6). PMID: 18510512    Free PMC article.
Computer-assisted assessment of the human epidermal growth factor receptor 2 immunohistochemical assay in imaged histologic sections using a membrane isolation algorithm and quantitative analysis of positive controls.
Bonnie H Hall, Monica Ianosi-Irimie, +3 authors, David J Foran.
BMC Med Imaging, 2008 Jun 07; 8. PMID: 18534031    Free PMC article.
Virtual microscopy as an enabler of automated/quantitative assessment of protein expression in TMAs.
Catherine Conway, Lynne Dobson, +3 authors, Daniel O'Shea.
Histochem Cell Biol, 2008 Aug 06; 130(3). PMID: 18679705
Automated quantitative assessment of HER-2/neu immunohistochemical expression in breast cancer.
Hela Masmoudi, Stephen M Hewitt, +2 authors, Marios A Gavrielides.
IEEE Trans Med Imaging, 2009 Jan 24; 28(6). PMID: 19164073    Free PMC article.
Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future.
Ronald S Weinstein, Anna R Graham, +7 authors, John R Gilbertson.
Hum Pathol, 2009 Jun 26; 40(8). PMID: 19552937
Digital pathology: exploring its applications in diagnostic surgical pathology practice.
Ana Richelia Jara-Lazaro, Thomas Paulraj Thamboo, Ming Teh, Puay Hoon Tan.
Pathology, 2010 Sep 22; 42(6). PMID: 20854068
Observer variability in the interpretation of HER2/neu immunohistochemical expression with unaided and computer-aided digital microscopy.
Marios A Gavrielides, Brandon D Gallas, +2 authors, Stephen M Hewitt.
Arch Pathol Lab Med, 2011 Feb 03; 135(2). PMID: 21284444    Free PMC article.
Reading immunohistochemical slides on a computer monitor--a multisite performance study using 180 HER2-stained breast carcinomas.
Aziza Nassar, Cynthia Cohen, +6 authors, Momin T Siddiqui.
Appl Immunohistochem Mol Morphol, 2011 Apr 09; 19(3). PMID: 21475038
Digital pathology: current status and future perspectives.
Shaimaa Al-Janabi, André Huisman, Paul J Van Diest.
Histopathology, 2011 Apr 12; 61(1). PMID: 21477260
Trainable immunohistochemical HER2/neu image analysis: a multisite performance study using 260 breast tissue specimens.
Aziza Nassar, Cynthia Cohen, +10 authors, Momin T Siddiqui.
Arch Pathol Lab Med, 2011 Jul 08; 135(7). PMID: 21732780
Assessment of Ki67 in breast cancer: recommendations from the International Ki67 in Breast Cancer working group.
Mitch Dowsett, Torsten O Nielsen, +19 authors, International Ki-67 in Breast Cancer Working Group.
J Natl Cancer Inst, 2011 Oct 01; 103(22). PMID: 21960707    Free PMC article.
Highly Cited.
Quantitative assessment and classification of tissue-based biomarker expression with color content analysis.
Brad Keller, Weijie Chen, Marios A Gavrielides.
Arch Pathol Lab Med, 2012 May 01; 136(5). PMID: 22540303
Concordance between whole-slide imaging and light microscopy for routine surgical pathology.
W Scott Campbell, Subodh M Lele, +3 authors, Steven H Hinrichs.
Hum Pathol, 2012 May 18; 43(10). PMID: 22591912
Comparing virtual with conventional microscopy for the consensus diagnosis of Barrett's neoplasia in the AspECT Barrett's chemoprevention trial pathology audit.
D S A Sanders, H Grabsch, +8 authors, AspECT Trial Management Group andTrial Principal Investigators.
Histopathology, 2012 Jun 22; 61(5). PMID: 22716297
Diagnostic reproducibility of tumour budding in colorectal cancer: a multicentre, multinational study using virtual microscopy.
Giacomo Puppa, Carlo Senore, +15 authors, Mauro Risio.
Histopathology, 2012 Jul 07; 61(4). PMID: 22765314
Evaluation of immunohistochemical staining using whole-slide imaging for HER2 scoring of breast cancer in comparison with real glass slides.
Yuzuru Kondo, Tatsuo Iijima, Masayuki Noguchi.
Pathol Int, 2012 Aug 29; 62(9). PMID: 22924845
Evaluation of whole slide imaging for routine surgical pathology: Looking through a broader scope.
Walter H Henricks.
J Pathol Inform, 2012 Dec 19; 3. PMID: 23248760    Free PMC article.
Validation of whole slide imaging for primary diagnosis in surgical pathology.
Thomas W Bauer, Lynn Schoenfield, +3 authors, Walter H Henricks.
Arch Pathol Lab Med, 2013 Jan 18; 137(4). PMID: 23323732
Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center.
Liron Pantanowitz, John H Sinard, +8 authors, College of American Pathologists Pathology and Laboratory Quality Center.
Arch Pathol Lab Med, 2013 May 03; 137(12). PMID: 23634907    Free PMC article.
Highly Cited. Review.
Comparison of virtual microscopy and glass slide microscopy among dermatology residents during a simulated in-training examination.
Katherine E Brick, Jason C Sluzevich, +3 authors, Carilyn N Wieland.
J Cutan Pathol, 2013 Jul 03; 40(9). PMID: 23808673
Production of a mouse monoclonal antibody reactive with a human nuclear antigen associated with cell proliferation.
J Gerdes, U Schwab, H Lemke, H Stein.
Int J Cancer, 1983 Jan 15; 31(1). PMID: 6339421
Highly Cited.
Quantitative immunohistochemistry using the CAS 200/486 image analysis system in invasive breast carcinoma: a reproducibility study.
S V Makkink-Nombrado, J P Baak, +2 authors, T van der Aa.
Anal Cell Pathol, 1995 Apr 01; 8(3). PMID: 7547496
Telepathology diagnosis by means of digital still images: an international validation study.
D S Weinberg, F A Allaert, +7 authors, M Doolittle.
Hum Pathol, 1996 Feb 01; 27(2). PMID: 8617451
The use of digital images in pathology.
P N Furness.
J Pathol, 1998 Jan 10; 183(3). PMID: 9422979
A scoring system for immunohistochemical staining: consensus report of the task force for basic research of the EORTC-GCCG. European Organization for Research and Treatment of Cancer-Gynaecological Cancer Cooperative Group.
P J van Diest, P van Dam, +4 authors, I Vergote.
J Clin Pathol, 1998 Feb 14; 50(10). PMID: 9462258    Free PMC article.
Tissue microarrays for high-throughput molecular profiling of tumor specimens.
J Kononen, L Bubendorf, +7 authors, O P Kallioniemi.
Nat Med, 1998 Jul 14; 4(7). PMID: 9662379
Highly Cited.
Evaluation environment for digital and analog pathology: a platform for validation studies.
Brandon D Gallas, Marios A Gavrielides, +5 authors, Stephen M Hewitt.
J Med Imaging (Bellingham), 2015 Jul 15; 1(3). PMID: 26158076    Free PMC article.
Morphology in the Digital Age: Integrating High-Resolution Description of Structural Alterations With Phenotypes and Genotypes.
Cynthia C Nast, Kevin V Lemley, +4 authors, Laura Barisoni.
Semin Nephrol, 2015 Jul 29; 35(3). PMID: 26215864    Free PMC article.
Comparison of the manual, semiautomatic, and automatic selection and leveling of hot spots in whole slide images for Ki-67 quantification in meningiomas.
Zaneta Swiderska, Anna Korzynska, +6 authors, Bartlomiej Grala.
Anal Cell Pathol (Amst), 2015 Aug 05; 2015. PMID: 26240787    Free PMC article.
Reproducibility of the NEPTUNE descriptor-based scoring system on whole-slide images and histologic and ultrastructural digital images.
Laura Barisoni, Jonathan P Troost, +18 authors, Stephen M Hewitt.
Mod Pathol, 2016 Apr 23; 29(7). PMID: 27102348    Free PMC article.
Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer.
Michel E Vandenberghe, Marietta L J Scott, +3 authors, Craig Barker.
Sci Rep, 2017 Apr 06; 7. PMID: 28378829    Free PMC article.
Digital pathology imaging as a novel platform for standardization and globalization of quantitative nephropathology.
Laura Barisoni, Charlotte Gimpel, +8 authors, Stephen M Hewitt.
Clin Kidney J, 2017 Jun 07; 10(2). PMID: 28584625    Free PMC article.
Influence of study design on digital pathology image quality evaluation: the need to define a clinical task.
Ljiljana Platiša, Leen Van Brantegem, +2 authors, Wilfried Philips.
J Med Imaging (Bellingham), 2017 Jun 28; 4(2). PMID: 28653011    Free PMC article.
Pitfalls in assessing stromal tumor infiltrating lymphocytes (sTILs) in breast cancer.
Zuzana Kos, Elvire Roblin, +91 authors, International Immuno-Oncology Biomarker Working Group.
NPJ Breast Cancer, 2020 May 16; 6. PMID: 32411819    Free PMC article.
A cohort study and meta-analysis of the evidence for consideration of Lauren subtype when prescribing adjuvant or palliative chemotherapy for gastric cancer.
Kunning Wang, Enxiao Li, +7 authors, Alex Boussioutas.
Ther Adv Med Oncol, 2020 Aug 06; 12. PMID: 32754227    Free PMC article.
Agreement in Histological Assessment of Mitotic Activity Between Microscopy and Digital Whole Slide Images Informs Conversion for Clinical Diagnosis.
Bih-Rong Wei, Charles H Halsey, +21 authors, R Mark Simpson.
Acad Pathol, 2019 Jul 20; 6. PMID: 31321298    Free PMC article.
Comparing Deep Learning and Immunohistochemistry in Determining the Site of Origin for Well-Differentiated Neuroendocrine Tumors.
Jordan Redemann, Fred A Schultz, +4 authors, Joshua A Hanson.
J Pathol Inform, 2020 Dec 22; 11. PMID: 33343993    Free PMC article.
Selection of Representative Histologic Slides in Interobserver Reproducibility Studies: Insights from Expert Review for Ovarian Carcinoma Subtype Classification.
Marios A Gavrielides, Brigitte M Ronnett, +2 authors, Jeffrey D Seidman.
J Pathol Inform, 2021 May 21; 12. PMID: 34012719    Free PMC article.
CGAT: Cell Graph ATtention Network for Grading of Pancreatic Disease Histology Images.
Mayank Baranwal, Santhoshi Krishnan, +2 authors, Arvind Rao.
Front Immunol, 2021 Oct 22; 12. PMID: 34671349    Free PMC article.