Journal Article
. 2003 May; 27(6):805-10.
doi: 10.1097/00000478-200306000-00012.

International variation in histologic grading is large, and persistent feedback does not improve reproducibility

Peter N Furness 1 Nicholas Taub  Karel J M Assmann  Giovanni Banfi  Jean-Pierre Cosyns  Anthony M Dorman  Claire M Hill  Silke K Kapper  Rudiger Waldherr  Aryvdas Laurinavicius  Niels Marcussen  Anna Paula Martins  Malfada Nogueira  Heinz Regele  Daniel Seron  Marta Carrera  Ståle Sund  Eero I Taskinen  Timo Paavonen  Tatjana Tihomirova  Rafail Rosenthal  
  • PMID: 12766585
  •     44 citations


Histologic grading systems are used to guide diagnosis, therapy, and audit on an international basis. The reproducibility of grading systems is usually tested within small groups of pathologists who have previously worked or trained together. This may underestimate the international variation of scoring systems. We therefore evaluated the reproducibility of an established system, the Banff classification of renal allograft pathology, throughout Europe. We also sought to improve reproducibility by providing individual feedback after each of 14 small groups of cases. Kappa values for all features studied were lower than any previously published, confirming that international variation is greater than interobserver variation as previously assessed. A prolonged attempt to improve reproducibility, using numeric or graphical feedback, failed to produce any detectable improvement. We then asked participants to grade selected photographs, to eliminate variation induced by pathologists viewing different areas of the slide. This produced improved kappa values only for some features. Improvement was influenced by the nature of the grade definitions. Definitions based on "area affected" by a process were not improved. The results indicate the danger of basing decisions on grading systems that may be applied very differently in different institutions.

Renal transplant biopsy specimen adequacy in a paediatric population.
Anne M Durkan, T James Beattie, +2 authors, Ian J Ramage.
Pediatr Nephrol, 2005 Oct 28; 21(2). PMID: 16252098
The proteogenomic path towards biomarker discovery.
Tara K Sigdel, Minnie M Sarwal.
Pediatr Transplant, 2008 Sep 04; 12(7). PMID: 18764911    Free PMC article.
Deconvoluting the 'omics' for organ transplantation.
Minnie M Sarwal.
Curr Opin Organ Transplant, 2009 Aug 01; 14(5). PMID: 19644370    Free PMC article.
A gene expression profile for detection of sufficient tumour cells in breast tumour tissue: microarray diagnosis eligibility.
Paul Roepman, Arenda Schuurman, +3 authors, Annuska M Glas.
BMC Med Genomics, 2009 Aug 14; 2. PMID: 19674463    Free PMC article.
Digital pathology evaluation of complement C4d component deposition in the kidney allograft biopsies is a useful tool to improve reproducibility of the scoring.
Ernesta Brazdziute, Arvydas Laurinavicius.
Diagn Pathol, 2011 Apr 22; 6 Suppl 1. PMID: 21489200    Free PMC article.
A data model and database for high-resolution pathology analytical image informatics.
Fusheng Wang, Jun Kong, +10 authors, Joel Saltz.
J Pathol Inform, 2011 Aug 17; 2. PMID: 21845230    Free PMC article.
Identification of biomarkers to assess organ quality and predict posttransplantation outcomes.
Mariano J Scian, Daniel G Maluf, +7 authors, Valeria R Mas.
Transplantation, 2012 Sep 21; 94(8). PMID: 22992769    Free PMC article.
EndoPredict improves the prognostic classification derived from common clinical guidelines in ER-positive, HER2-negative early breast cancer.
P Dubsky, M Filipits, +17 authors, Austrian Breast and Colorectal Cancer Study Group (ABCSG).
Ann Oncol, 2012 Oct 05; 24(3). PMID: 23035151    Free PMC article.
Challenges and solutions in the development of genomic biomarker panels: a systematic phased approach.
K Shahzad, A Fatima, +5 authors, Mc Deng.
Curr Genomics, 2012 Dec 04; 13(4). PMID: 23204923    Free PMC article.
The predictive value of kidney allograft baseline biopsies for long-term graft survival.
Katrien De Vusser, Evelyne Lerut, +5 authors, Maarten Naesens.
J Am Soc Nephrol, 2013 Aug 21; 24(11). PMID: 23949799    Free PMC article.
Transplantation immunology in 2013: New approaches to diagnosis of rejection.
Nicholas A Zwang, Laurence A Turka.
Nat Rev Nephrol, 2013 Dec 18; 10(2). PMID: 24342957
Banff Initiative for Quality Assurance in Transplantation (BIFQUIT): reproducibility of polyomavirus immunohistochemistry in kidney allografts.
B Adam, P Randhawa, +6 authors, M Mengel.
Am J Transplant, 2014 Aug 06; 14(9). PMID: 25091177    Free PMC article.
Disappearance of T Cell-Mediated Rejection Despite Continued Antibody-Mediated Rejection in Late Kidney Transplant Recipients.
Philip F Halloran, Jessica Chang, +11 authors, Jeff Reeve.
J Am Soc Nephrol, 2014 Nov 08; 26(7). PMID: 25377077    Free PMC article.
Interobserver agreement on histopathological lesions in class III or IV lupus nephritis.
Suzanne Wilhelmus, H Terence Cook, +4 authors, Ingeborg M Bajema.
Clin J Am Soc Nephrol, 2014 Nov 12; 10(1). PMID: 25384443    Free PMC article.
The Revisited Classification of GN in SLE at 10 Years: Time to Re-Evaluate Histopathologic Lesions.
Suzanne Wilhelmus, Charles E Alpers, +8 authors, Ingeborg M Bajema.
J Am Soc Nephrol, 2015 Jul 15; 26(12). PMID: 26152271    Free PMC article.
What is the best way to measure renal fibrosis?: A pathologist's perspective.
Alton B Farris, Charles E Alpers.
Kidney Int Suppl (2011), 2015 Aug 28; 4(1). PMID: 26312144    Free PMC article.
The Use of Genomics and Pathway Analysis in Our Understanding and Prediction of Clinical Renal Transplant Injury.
Madhav C Menon, Karen L Keung, Barbara Murphy, Philip J OʼConnell.
Transplantation, 2015 Oct 09; 100(7). PMID: 26447506    Free PMC article.
Molecular assessment of disease states in kidney transplant biopsy samples.
Philip F Halloran, Konrad S Famulski, Jeff Reeve.
Nat Rev Nephrol, 2016 Jun 28; 12(9). PMID: 27345248
From High-Throughput Microarray-Based Screening to Clinical Application: The Development of a Second Generation Multigene Test for Breast Cancer Prognosis.
Jan C Brase, Ralf Kronenwett, +2 authors, Marcus Schmidt.
Microarrays (Basel), 2013 Jan 01; 2(3). PMID: 27605191    Free PMC article.
Multiparametric Functional MRI: A Tool to Uncover Subtle Changes following Allogeneic Renal Transplantation.
Mike Notohamiprodjo, Aivars Kalnins, +8 authors, Joachim Andrassy.
PLoS One, 2016 Nov 08; 11(11). PMID: 27820833    Free PMC article.
The Effect of Cortex/Medulla Proportions on Molecular Diagnoses in Kidney Transplant Biopsies: Rejection and Injury Can Be Assessed in Medulla.
K S Madill-Thomsen, R C Wiggins, +2 authors, P F Halloran.
Am J Transplant, 2017 Feb 23; 17(8). PMID: 28226404    Free PMC article.
Digital pathology imaging as a novel platform for standardization and globalization of quantitative nephropathology.
Laura Barisoni, Charlotte Gimpel, +8 authors, Stephen M Hewitt.
Clin Kidney J, 2017 Jun 07; 10(2). PMID: 28584625    Free PMC article.
Assessing rejection-related disease in kidney transplant biopsies based on archetypal analysis of molecular phenotypes.
Jeff Reeve, Georg A Böhmig, +5 authors, MMDx-Kidney study group.
JCI Insight, 2017 Jun 15; 2(12). PMID: 28614805    Free PMC article.
Biomarkers to detect rejection after kidney transplantation.
Vikas R Dharnidharka, Andrew Malone.
Pediatr Nephrol, 2017 Jun 21; 33(7). PMID: 28631040
Development of CD3 cell quantitation algorithms for renal allograft biopsy rejection assessment utilizing open source image analysis software.
Andres Moon, Geoffrey H Smith, +3 authors, Alton B Brad Farris.
Virchows Arch, 2017 Nov 09; 472(2). PMID: 29116389
Histological findings to five years after early conversion of kidney transplant patients from cyclosporine to everolimus: an analysis from the randomized ZEUS study.
Ute Eisenberger, Klemens Budde, +13 authors, ZEUS Study Investigators.
BMC Nephrol, 2018 Jun 30; 19(1). PMID: 29954336    Free PMC article.
Pathology of Calcineurin and Mammalian Target of Rapamycin Inhibitors in Kidney Transplantation.
Rita Leal, Demetra Tsapepas, +3 authors, Ibrahim Batal.
Kidney Int Rep, 2018 Oct 03; 3(2). PMID: 30276344    Free PMC article.
Pre-implantation kidney biopsy: value of the expertise in determining histological score and comparison with the whole organ on a series of discarded kidneys.
Ilaria Girolami, Giovanni Gambaro, +14 authors, Albino Eccher.
J Nephrol, 2019 Sep 01; 33(1). PMID: 31471818
A method to reduce variability in scoring antibody-mediated rejection in renal allografts: implications for clinical trials - a retrospective study.
Byron Smith, Lynn D Cornell, +9 authors, Mark Stegall.
Transpl Int, 2018 Sep 05; 32(2). PMID: 30179275    Free PMC article.
Reliability of deceased-donor procurement kidney biopsy images uploaded in United Network for Organ Sharing.
Sherry G Mansour, Isaac E Hall, +10 authors, Chirag R Parikh.
Clin Transplant, 2018 Nov 06; 32(12). PMID: 30387908    Free PMC article.
Histological characteristics of Acute Tubular Injury during Delayed Graft Function predict renal function after renal transplantation.
Tobias T Pieters, Lucas L Falke, +8 authors, Maarten B Rookmaaker.
Physiol Rep, 2019 Mar 02; 7(5). PMID: 30821122    Free PMC article.
Profiling of mRNA of interstitial fibrosis and tubular atrophy with subclinical inflammation in recipients after kidney transplantation.
Qiang Fu, Minxue Liao, +7 authors, Shaoping Deng.
Aging (Albany NY), 2019 Jul 26; 11(14). PMID: 31343413    Free PMC article.
Transcriptome Analysis in Renal Transplant Biopsies Not Fulfilling Rejection Criteria.
Francesc Moreso, Joana Sellarès, María José Soler, Daniel Serón.
Int J Mol Sci, 2020 Mar 28; 21(6). PMID: 32213927    Free PMC article.
Usefulness of morphometric image analysis with Sirius Red to assess interstitial fibrosis after renal transplantation from uncontrolled circulatory death donors.
Myriam Dao, Christelle Pouliquen, +6 authors, Sophie Ferlicot.
Sci Rep, 2020 Apr 25; 10(1). PMID: 32327683    Free PMC article.
Single-cell Transcriptomics and Solid Organ Transplantation.
Andrew F Malone, Benjamin D Humphreys.
Transplantation, 2019 Apr 05; 103(9). PMID: 30946217    Free PMC article.
Extracellular Matrix in Kidney Fibrosis: More Than Just a Scaffold.
Roman David Bülow, Peter Boor.
J Histochem Cytochem, 2019 May 23; 67(9). PMID: 31116062    Free PMC article.
EndoPredict® in early hormone receptor-positive, HER2-negative breast cancer.
K Almstedt, S Mendoza, +9 authors, M Schmidt.
Breast Cancer Res Treat, 2020 May 22; 182(1). PMID: 32436145    Free PMC article.
Magnetic Resonance Imaging for Evaluation of Interstitial Fibrosis in Kidney Allografts.
Andrea Beck-Tölly, Michael Eder, +8 authors, Georg A Böhmig.
Transplant Direct, 2020 Nov 03; 6(8). PMID: 33134501    Free PMC article.
Molecular Diversity of Clinically Stable Human Kidney Allografts.
Dmitry Rychkov, Swastika Sur, Marina Sirota, Minnie M Sarwal.
JAMA Netw Open, 2021 Jan 26; 4(1). PMID: 33492376    Free PMC article.
Use of the Tissue Common Rejection Module Score in Kidney Transplant as an Objective Measure of Allograft Inflammation.
Arya Zarinsefat, Jose M Arreola Guerra, +13 authors, Minnie M Sarwal.
Front Immunol, 2021 Feb 23; 11. PMID: 33613539    Free PMC article.
Automated Computational Detection of Interstitial Fibrosis, Tubular Atrophy, and Glomerulosclerosis.
Brandon Ginley, Kuang-Yu Jen, +13 authors, Pinaki Sarder.
J Am Soc Nephrol, 2021 Feb 25;. PMID: 33622976    Free PMC article.
Microvascular inflammation in renal allograft biopsies assessed by endothelial and leukocyte co-immunostain: a retrospective study on reproducibility and clinical/prognostic correlates.
Marco Delsante, Umberto Maggiore, +7 authors, Avi Z Rosenberg.
Transpl Int, 2018 Nov 06; 32(3). PMID: 30395360    Free PMC article.
Image Analysis Pipeline for Renal Allograft Evaluation and Fibrosis Quantification.
Alton Brad Farris, Juan Vizcarra, +3 authors, Julien Hogan.
Kidney Int Rep, 2021 Jul 27; 6(7). PMID: 34307982    Free PMC article.
Data-driven Derivation and Validation of Novel Phenotypes for Acute Kidney Transplant Rejection using Semi-supervised Clustering.
Thibaut Vaulet, Gillian Divard, +16 authors, Maarten Naesens.
J Am Soc Nephrol, 2021 Mar 11; 32(5). PMID: 33687976    Free PMC article.