FREE SHIPPING over $40
OVER 9000 FIVE-STAR REVIEWS
5% OFF ALL SUBSCRIPTIONS
100% HAPPINESS GUARANTEE
Analyze Diet
0
Home / Equine Research Bank / J Vet Diagn Invest / IIB or not IIB, part 2: assessing inter-rater and intra-rate...
Journal of veterinary diagnostic investigation : official publication of the American Association of Veterinary Laboratory Diagnosticians, Inc2021; 34(2); 215-225; doi: 10.1177/10406387211062866

IIB or not IIB, part 2: assessing inter-rater and intra-rater repeatability of the Kenney-Doig scale in equine endometrial biopsy evaluation.

Abstract: Inter- and intra-rater variability negatively affects the reliability of various histopathology grading scales used as prognostic aids in human and veterinary medicine. The Kenney-Doig categorization (grading) scale, which is used to associate equine endometrial histologic lesions with prognostic estimation of a broodmare's reproductive potential, has not been evaluated for inter- or intra-rater variability, to our knowledge. To assess whether the Kenney-Doig system produces reliable results among observers, 8 pathologists, all with American College of Veterinary Pathologists certification, were recruited to blindly categorize the same set of 63 digital equine endometrial biopsy slides as well as to re-evaluate anonymously 21 of 63 of these slides at a later time. Cohen kappa values for pairwise comparison of final Kenney-Doig categories were -0.05 to 0.46 (unweighted) and 0.08-0.64 (weighted), with an average Light kappa of 0.19 (unweighted) and 0.36 (weighted) across all 8 pathologists, 0.14 (unweighted) and 0.33 (weighted) for pathologists at different institutions, and 0.22 (unweighted) and 0.46 (weighted) for pathologists at the same institution. Intra-class correlations measuring intra-rater agreement were 0.12-0.77 with an average of 0.55 for all 8 pathologists. We found that only slight-to-moderate inter-rater agreement and poor-to-good intra-rater agreement was produced by 8 pathologists using the Kenney-Doig scale, suggesting that the system is subject to significant observer variability and care should be taken when communicating Kenney-Doig categories to submitting clinicians with emphasis on the quality of endometrial lesions present instead of the category and associated expected foaling rate.
Get Full Text
Publication Date: 2021-12-29 PubMed ID: 34965793PubMed Central: PMC8921799DOI: 10.1177/10406387211062866Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
LinkedInCopy
  • Journal Article

Summary

This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.

The researchers tested the reliability of the Kenney-Doig grading system, used to rate equine endometrial lesions for predicting a broodmare’s reproductive potential. Their findings indicate that different pathologists’ ratings using this scale can vary significantly, suggesting a need for cautious communication regarding the system’s categories and expected foaling rate.

Study Objective

  • The study aimed to test the reliability and consistency of the Kenney-Doig scale which is used in veterinary medicine to assess equine endometrial lesions. This rating system serves as a prognostic tool estimating the reproductive potential of a broodmare. The reliability of this system has not been evaluated before in terms of consistency across different raters (inter-rater) or consistency of the same rater over time (intra-rater).

Methodology

  • Eight pathologists, all certified by the American College of Veterinary Pathologists, were recruited for this study. They were asked to provide blind categorization using the Kenney-Doig scale on a set of 63 digital equine endometrial biopsy slides.
  • In order to evaluate intra-rater variability, 21 slides from the original set were re-evaluated by the same pathologists at a later stage.
  • The agreement among the pathologists for the initial and repeat assessments was then measured using Cohen kappa values and Intra-class correlations.

Key Findings

  • The Cohen kappa values indicated broad variation in inter-rater compatibility, ranging from -0.05 to 0.46 (unweighted) and 0.08-0.64 (weighted).
  • The consistency between the results of the same pathologist at different times (intra-rater consistency) as evaluated by intra-class correlations, ranged from 0.12 to 0.77, with an average of 0.55 for all eight pathologists.
  • These findings suggest that there is only slight to moderate agreement between different observers, and poor to good agreement when the same observer rates at different times.

Conclusion

  • The results indicate a high degree of variability in the use of the Kenney-Doig scale, pointing to a risk of inconsistency in its application.
  • This highlights the need for caution when interpreting and communicating findings using this system, with a recommendation to focus more on the quality of endometrial lesions instead of the category and its associated expected foaling rate.

Cite This Article

APA
Westendorf J, Wobeser B, Epp T. (2021). IIB or not IIB, part 2: assessing inter-rater and intra-rater repeatability of the Kenney-Doig scale in equine endometrial biopsy evaluation. J Vet Diagn Invest, 34(2), 215-225. https://doi.org/10.1177/10406387211062866

Publication

Journal of veterinary diagnostic investigation : official publication of the American Association of Veterinary Laboratory Diagnosticians, Inc
ISSN: 1943-4936
NlmUniqueID: 9011490
Country: United States
Language: English
Volume: 34
Issue: 2
Pages: 215-225

Researcher Affiliations

Westendorf, Jane
  • Department of Veterinary Pathology, Western College of Veterinary Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada.
Wobeser, Bruce
  • Department of Veterinary Pathology, Western College of Veterinary Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada.
Epp, Tasha
  • Department of Veterinary Pathology, Western College of Veterinary Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada.

MeSH Terms

  • Animals
  • Biopsy / veterinary
  • Endometrium
  • Female
  • Horses
  • Humans
  • Observer Variation
  • Reproducibility of Results

Conflict of Interest Statement

Declaration of conflicting interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

This article includes 54 references
  1. Allen WR. The diagnosis and handling of early gestational abnormalities in the mare.. Anim Reprod Sci 1992;28:31–38.
  2. Aupperle H. Physiological and pathological expression of intermediate filaments in the equine endometrium.. Res Vet Sci 2004;76:249–255.
    pubmed: 15046960
  3. Bergeron C. A multicentric European study testing the reproducibility of the WHO classification of endometrial hyperplasia with a proposal of a simplified working classification for biopsy and curettage specimens.. Am J Surg Pathol 1999;23:1102–1108.
    pubmed: 10478671
  4. Brothwell DJ. Observer agreement in the grading of oral epithelial dysplasia.. Community Dent Oral Epidemiol 2003;31:300–305.
    pubmed: 12846853
  5. Bujang MA, Baharum N. Guidelines of the minimum sample size requirements for Cohen’s kappa.. Epidemiol Biostat Public Health 2017;14:e12267-1–e12267-10.
  6. Cross SS. Kappa statistics as indicators of quality assurance in histopathology and cytopathology.. J Clin Pathol 1996;49:597–599.
    pmc: PMC500580pubmed: 8813964
  7. Cross SS. Grading and scoring in histopathology.. Histopathology 1998;33:99–106.
    pubmed: 9762541
  8. de la Concha-Bermejillo A, Kennedy PC. Prognostic value of endometrial biopsy in the mare: a retrospective analysis.. J Am Vet Med Assoc 1982;181:680–681.
    pubmed: 7141963
  9. de Vet HC. Interobserver variation in histopathological grading of cervical dysplasia.. J Clin Epidemiol 1990;43:1395–1398.
    pubmed: 2254777
  10. de Vet HC. Efforts to improve interobserver agreement in histopathological grading.. J Clin Epidemiol 1995;48:869–873.
    pubmed: 7782794
  11. Dellon ES. Inter- and intraobserver reliability and validation of a new method for determination of eosinophil counts in patients with esophageal eosinophilia.. Dig Dis Sci 2010;55:1940–1949.
    pmc: PMC2895956pubmed: 19830560
  12. Evans TJ. Morphometric analysis of endometrial periglandular fibrosis in mares.. Am J Vet Res 1998;59:1209–1214.
    pubmed: 9781449
  13. Fadare O. The diagnosis of endometrial carcinomas with clear cells by gynecologic pathologists: an assessment of interobserver variability and associated morphologic features.. Am J Surg Pathol 2012;36:1107–1118.
    pubmed: 22790851
  14. Grether JK. Reliability of placental histology using archived specimens.. Paediatr Perinat Epidemiol 1999;13:489–495.
    pubmed: 10563368
  15. Hoffmann C. The equine endometrosis: new insights into the pathogenesis.. Anim Reprod Sci 2009;111:261–278.
    pubmed: 18468817
  16. Ishak K. Histological grading and staging of chronic hepatitis.. J Hepatol 1995;22:696–699.
    pubmed: 7560864
  17. Kenney R, Doig P. Equine endometrial biopsy.. In: Morrow D, ed. Current Therapy in Theriogenology: Diagnosis, Treatment, and Prevention of Reproductive Diseases in Small & Large Animals. 2nd ed. Saunders, 1986:723–729.
  18. Kenney RM. Cyclic and pathologic changes of the mare endometrium as detected by biopsy, with a note on early embryonic death.. J Am Vet Med Assoc 1978;172:241–262.
    pubmed: 621166
  19. Kiupel M. Proposal of a 2-tier histologic grading system for canine cutaneous mast cell tumors to more accurately predict biological behavior.. Vet Pathol 2011;48:147–155.
    pmc: PMC8369849pubmed: 21062911
  20. Koelink PJ. Development of reliable, valid and responsive scoring systems for endoscopy and histology in animal models for inflammatory bowel disease.. J Crohns Colitis 2018;12:794–803.
    pmc: PMC6022651pubmed: 29608662
  21. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research.. J Chiropr Med 2016;15:155–163.
    pmc: PMC4913118pubmed: 27330520
  22. Landis JR, Koch GG. The measurement of observer agreement for categorical data.. Biometrics 1977;33:159–174.
    pubmed: 843571
  23. Leblanc MM. When to refer an infertile mare to a theriogenologist.. Theriogenology 2008;70:421–429.
    pubmed: 18514809
  24. Lehmann J. Morpho-functional studies regarding the fertility prognosis of mares suffering from equine endometrosis.. Theriogenology 2011;76:1326–1336.
    pubmed: 21855986
  25. Light RJ. Measures of response agreement for qualitative data: some generalizations and alternatives.. Psychol Bull 1971;76:365–377.
  26. Love C. Techniques in reproductive examination: endometrial biopsy.. In: McKinnon AO, et al.., eds. Equine Reproduction. 2nd ed. Wiley Blackwell, 2011:1929–1939.
  27. Lunelli D. Collagen types in relation to expression of estradiol and progesterone receptors in equine endometrial fibrosis.. Adv Biosci Biotechnol 2013;4:599–605.
  28. Malpica A. Kappa statistics to measure interrater and intrarater agreement for 1790 cervical biopsy specimens among twelve pathologists: qualitative histopathologic analysis and methodologic issues.. Gynecol Oncol 2005;99(3 Suppl 1):S38–S52.
    pubmed: 16183106
  29. Mambelli LI. Changes in expression pattern of selected endometrial proteins following mesenchymal stem cells infusion in mares with endometrosis.. PLoS One 2014;9:e97889.
    pmc: PMC4046935pubmed: 24901368
  30. Manning S. A prospective study of components of the breeding soundness evaluation, including endometrial biopsy, in mares selected for fertility.. Unpublished master’s thesis. Western College of Veterinary Medicine, University of Saskatchewan, 2002.
  31. McCue P. Clinical Equine Reproduction: Anatomy, Physiology, Pathology, and Breeding Management.. Vol. 1. Colorado State University, 2019.
  32. McHugh ML. Interrater reliability: the kappa statistic.. Biochem Med (Zagreb) 2012;22:276–282.
    pmc: PMC3900052pubmed: 23092060
  33. Minkwitz C. Plasticity of endometrial epithelial and stromal cells—a new approach towards the pathogenesis of equine endometrosis.. Reprod Domest Anim 2019;54:835–845.
    pubmed: 30907027
  34. Munkedal DLE. Significant individual variation between pathologists in the evaluation of colon cancer specimens after complete mesocolic excision.. Dis Colon Rectum 2016;59:953–961.
    pubmed: 27602926
  35. Northrup NC. Variation among pathologists in the histologic grading of canine cutaneous mast cell tumors with uniform use of a single grading reference.. J Vet Diagn Invest 2005;17:561–564.
    pubmed: 16475514
  36. Northrup NC. Variation among pathologists in histologic grading of canine cutaneous mast cell tumors.. J Vet Diagn Invest 2005;17:245–248.
    pubmed: 15945380
  37. Oddsdóttir C. Development of endometrial fibrosis in the mare: factors involved in tissue remodelling and collagen deposition.. Doctoral dissertation. Royal (Dick) School of Veterinary Studies, 2007.
  38. Ricketts SW, Alonso S. Assessment of the breeding prognosis of mares using paired endometrial biopsy techniques.. Equine Vet J 1991;23:185–188.
    pubmed: 1884698
  39. Ricketts SW, Alonso S. The effect of age and parity on the development of equine chronic endometrial disease.. Equine Vet J 1991;23:189–192.
    pubmed: 1884699
  40. Robbins P. Histological grading of breast carcinomas: a study of interobserver agreement.. Hum Pathol 1995;26:873–879.
    pubmed: 7635449
  41. Schlafer DH. Equine endometrial biopsy: enhancement of clinical value by more extensive histopathology and application of new diagnostic techniques?. Theriogenology 2007;68:413–422.
    pubmed: 17560641
  42. Scholten AN. Prognostic significance and interobserver variability of histologic grading systems for endometrial carcinoma.. Cancer 2004;100:764–772.
    pubmed: 14770433
  43. Schöniger S, Schoon H-A. The healthy and diseased equine endometrium: a review of morphological features and molecular analyses.. Animals (Basel) 2020;10:625.
    pmc: PMC7222714pubmed: 32260515
  44. Schoon D. Angioses in the equine endometrium—pathogenesis and clinical correlations.. Pferdeheilkunde 1999;15:541–546.
  45. Schoon HA. The endometrial biopsy in the mare with regard to clinical correlations.. Pferdeheilkunde 1997;13:453–464.
  46. Schoon HA. “Endometrial maldifferentiation”—a clinically significant diagnosis in equine reproduction?. Pferdeheilkunde 1999;15:555–559.
  47. Schoon HA. Functional disturbances in the endometrium of barren mares a histological and immunohistological study.. J Reprod Fertil Suppl 2000;56:381–391.
    pubmed: 20681150
  48. Silcocks PB. Measuring repeatability and validity of histological diagnosis—a brief review with some practical examples.. J Clin Pathol 1983;36:1269–1275.
    pmc: PMC498544pubmed: 6313770
  49. Snider TA. Equine endometrial biopsy reviewed: observation, interpretation, and application of histopathologic data.. Theriogenology 2011;75:1567–1581.
    pubmed: 21356552
  50. Stanton MB. Endometrial cysts in the mare.. J Equine Vet Sci 2004;24:14–19.
  51. Stenkvist B. Histopathological systems of breast cancer classification: reproducibility and clinical significance.. J Clin Pathol 1983;36:392–398.
    pmc: PMC498233pubmed: 6833508
  52. Walter I. Association of endometrosis in horses with differentiation of periglandular myofibroblasts and changes of extracellular matrix proteins.. Reproduction 2001;121:581–586.
    pubmed: 11277878
  53. Warners MJ. Reliability of histologic assessment in patients with eosinophilic oesophagitis.. Aliment Pharmacol Ther 2018;47:940–950.
    pubmed: 29460418
  54. Westendorf J. IIB or not IIB, part 1: retrospective evaluation of Kenney–Doig categorization of equine endometrial biopsies at a veterinary diagnostic laboratory and comparison with published reports.. J Vet Diagn Invest 2021;34(2).
    pmc: PMC8921822pubmed: 34841986
Subscribe to our newsletter
Join 99,357 horse owners like you who receive our email updates on horse health and nutrition.