FREE SHIPPING over $40
OVER 9000 FIVE-STAR REVIEWS
5% OFF ALL SUBSCRIPTIONS
100% HAPPINESS GUARANTEE
Analyze Diet
0
Home / Equine Research Bank / Vet Pathol / Cytologic scoring of equine exercise-induced pulmonary hemor...
Veterinary pathology2022; 60(1); 75-85; doi: 10.1177/03009858221137582

Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm.

Abstract: Exercise-induced pulmonary hemorrhage (EIPH) is a relevant respiratory disease in sport horses, which can be diagnosed by examination of bronchoalveolar lavage fluid (BALF) cells using the total hemosiderin score (THS). The aim of this study was to evaluate the diagnostic accuracy and reproducibility of annotators and to validate a deep learning-based algorithm for the THS. Digitized cytological specimens stained for iron were prepared from 52 equine BALF samples. Ten annotators produced a THS for each slide according to published methods. The reference methods for comparing annotator's and algorithmic performance included a ground truth dataset, the mean annotators' THSs, and chemical iron measurements. Results of the study showed that annotators had marked interobserver variability of the THS, which was mostly due to a systematic error between annotators in grading the intracytoplasmatic hemosiderin content of individual macrophages. Regarding overall measurement error between the annotators, 87.7% of the variance could be reduced by using standardized grades based on the ground truth. The algorithm was highly consistent with the ground truth in assigning hemosiderin grades. Compared with the ground truth THS, annotators had an accuracy of diagnosing EIPH (THS of < or ≥ 75) of 75.7%, whereas, the algorithm had an accuracy of 92.3% with no relevant differences in correlation with chemical iron measurements. The results show that deep learning-based algorithms are useful for improving reproducibility and routine applicability of the THS. For THS by experts, a diagnostic uncertainty interval of 40 to 110 is proposed. THSs within this interval have insufficient reproducibility regarding the EIPH diagnosis.
Get Full Text
Publication Date: 2022-11-17 PubMed ID: 36384369PubMed Central: PMC9827485DOI: 10.1177/03009858221137582Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
LinkedInCopy
  • Journal Article
  • Research Support
  • Non-U.S. Gov't

Summary

This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.

The study revolves around assessing the performance of human experts and a deep learning algorithm in diagnosing a horse respiratory disease called exercise-induced pulmonary hemorrhage (EIPH), using a method known as the total hemosiderin score (THS). The research determined that the deep learning algorithm had a higher level of accuracy in diagnosing EIPH than human experts and was able to reduce variation in measurements.

Study Design

  • The study included bronchoalveolar lavage fluid (BALF) samples from 52 horses, which were examined and categorized for signs of EIPH.
  • These assessments were done by ten annotators or human experts, who determined a THS for each sample following standard methods.
  • Meanwhile, a deep learning algorithm was also programmed to assign a THS to each sample.
  • Three reference methods were used to gauge the efficacy of both the annotators and the algorithm: a ground truth dataset, mean scores of the annotators, and chemical iron measurements.

Annotators’ Performance

  • The researchers found that the human annotators displayed significant variability in their THS assignment, primarily caused by differing interpretations of the amount of hemosiderin content in individual macrophages.
  • By using standardized grades derived from the ground truth, the variation in measurements done by the annotators could be reduced by 87.7%.
  • Compared to the ground truth THS, annotators were 75.7% accurate in diagnosing EIPH.

Algorithm’s Performance

  • On the other hand, the deep learning algorithm demonstrated consistency with the ground truth when assigning hemosiderin grades.
  • Its accuracy in diagnosing EIPH was higher, at 92.3%, and showed no meaningful differences when compared to chemical iron measurements.

Conclusions and Recommendations

  • Based on these findings, the researchers surmise that deep learning algorithms can help enhance the reproducibility and routine use of the THS method in diagnosing EIPH.
  • They also propose a diagnostic uncertainty interval of 40 to 110 for expert-assigned THS, which suggests those scores that fall within this range are deemed to lack sufficient reproducibility in diagnosing EIPH.

Cite This Article

APA
Bertram CA, Marzahl C, Bartel A, Stayt J, Bonsembiante F, Beeler-Marfisi J, Barton AK, Brocca G, Gelain ME, Gläsel A, Preez KD, Weiler K, Weissenbacher-Lang C, Breininger K, Aubreville M, Maier A, Klopfleisch R, Hill J. (2022). Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm. Vet Pathol, 60(1), 75-85. https://doi.org/10.1177/03009858221137582

Publication

Veterinary pathology
ISSN: 1544-2217
NlmUniqueID: 0312020
Country: United States
Language: English
Volume: 60
Issue: 1
Pages: 75-85

Researcher Affiliations

Bertram, Christof A
  • University of Veterinary Medicine Vienna, Vienna, Austria.
  • Freie Universität Berlin, Berlin, Germany.
Marzahl, Christian
  • Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
  • EUROIMMUN Medizinische Labordiagnostika AG, Lübeck, Germany.
Bartel, Alexander
  • Freie Universität Berlin, Berlin, Germany.
Stayt, Jason
  • Novavet Diagnostics, Bayswater, Western Australia.
Bonsembiante, Federico
  • University of Padova, Legnaro, Italy.
Beeler-Marfisi, Janet
  • University of Guelph, Guelph, Ontario, Canada.
Barton, Ann K
  • Freie Universität Berlin, Berlin, Germany.
Brocca, Ginevra
  • University of Padova, Legnaro, Italy.
Gelain, Maria E
  • University of Padova, Legnaro, Italy.
Gläsel, Agnes
  • Justus-Liebig-Universität Giessen, Giessen, Germany.
Preez, Kelly du
  • University of Pretoria, Pretoria, South Africa.
Weiler, Kristina
  • Justus-Liebig-Universität Giessen, Giessen, Germany.
Weissenbacher-Lang, Christiane
  • University of Veterinary Medicine Vienna, Vienna, Austria.
Breininger, Katharina
  • Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
Aubreville, Marc
  • Technische Hochschule Ingolstadt, Ingolstadt, Germany.
Maier, Andreas
  • Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
Klopfleisch, Robert
  • Freie Universität Berlin, Berlin, Germany.
Hill, Jenny
  • Novavet Diagnostics, Bayswater, Western Australia.

MeSH Terms

  • Animals
  • Bronchoalveolar Lavage Fluid
  • Deep Learning
  • Hemorrhage / diagnosis
  • Hemorrhage / veterinary
  • Hemosiderin
  • Horse Diseases / diagnosis
  • Horses
  • Iron
  • Lung Diseases / diagnosis
  • Lung Diseases / veterinary
  • Reproducibility of Results

Conflict of Interest Statement

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

This article includes 36 references
  1. Abels E, Pantanowitz L, Aeffner F, Zarella MD, van der Laak J, Bui MM, Vemuri VN, Parwani AV, Gibbs J, Agosto-Arroyo E, Beck AH, Kozlowski C. Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the Digital Pathology Association.. J Pathol 2019 Nov;249(3):286-294.
    pmc: PMC6852275pubmed: 31355445doi: 10.1002/path.5331google scholar: lookup
  2. Aeffner F, Wilson K, Martin NT, Black JC, Hendriks CLL, Bolon B, Rudmann DG, Gianani R, Koegler SR, Krueger J, Young GD. The Gold Standard Paradox in Digital Image Analysis: Manual Versus Automated Scoring as Ground Truth.. Arch Pathol Lab Med 2017 Sep;141(9):1267-1275.
    pubmed: 28557614doi: 10.5858/arpa.2016-0386-ragoogle scholar: lookup
  3. Aubreville M, Bertram C, Klopfleisch R. SlideRunner. Bildverarbeitung für die Medizin 2018 2018:309–314.
  4. Aubreville M, Bertram CA, Marzahl C, Gurtner C, Dettwiler M, Schmidt A, Bartenschlager F, Merz S, Fragoso M, Kershaw O, Klopfleisch R, Maier A. Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region.. Sci Rep 2020 Oct 5;10(1):16447.
    pmc: PMC7536430pubmed: 33020510doi: 10.1038/s41598-020-73246-2google scholar: lookup
  5. Bates D, Mächler M, Bolker B. Fitting linear mixed-effects models using lme4. J Stat Soft 2015;67:1–48.
  6. Bertram CA, Aubreville M, Donovan TA, Bartel A, Wilm F, Marzahl C, Assenmacher CA, Becker K, Bennett M, Corner S, Cossic B, Denk D, Dettwiler M, Gonzalez BG, Gurtner C, Haverkamp AK, Heier A, Lehmbecker A, Merz S, Noland EL, Plog S, Schmidt A, Sebastian F, Sledge DG, Smedley RC, Tecilla M, Thaiwong T, Fuchs-Baumgartinger A, Meuten DJ, Breininger K, Kiupel M, Maier A, Klopfleisch R. Computer-assisted mitotic count using a deep learning-based algorithm improves interobserver reproducibility and accuracy.. Vet Pathol 2022 Mar;59(2):211-226.
    pmc: PMC8928234pubmed: 34965805doi: 10.1177/03009858211067478google scholar: lookup
  7. Bertram CA, Stathonikos N, Donovan TA, Bartel A, Fuchs-Baumgartinger A, Lipnik K, van Diest PJ, Bonsembiante F, Klopfleisch R. Validation of digital microscopy: Review of validation methods and sources of bias.. Vet Pathol 2022 Jan;59(1):26-38.
    pmc: PMC8761960pubmed: 34433345doi: 10.1177/03009858211040476google scholar: lookup
  8. Bertram CA, Klopfleisch R. The Pathologist 2.0: An Update on Digital Pathology in Veterinary Medicine.. Vet Pathol 2017 Sep;54(5):756-766.
    pubmed: 28578626doi: 10.1177/0300985817709888google scholar: lookup
  9. Center SA, McDonough SP, Bogdanovic L. Digital image analysis of rhodanine-stained liver biopsy specimens for calculation of hepatic copper concentrations in dogs.. Am J Vet Res 2013 Dec;74(12):1474-80.
    pubmed: 24274883doi: 10.2460/ajvr.74.12.1474google scholar: lookup
  10. Cian F, Monti P, Durham A. Cytology of the lower respiratory tract in horses: an updated review. Equine Vet Educ 2015;27:544–553.
  11. Crispe EJ, Lester GD, Secombe CJ, Perera DI. The association between exercise-induced pulmonary haemorrhage and race-day performance in Thoroughbred racehorses.. Equine Vet J 2017 Sep;49(5):584-589.
    pubmed: 28118491doi: 10.1111/evj.12671google scholar: lookup
  12. Depecker M, Couroucé-Malblanc A, Leleu C, Genneviève V, Pitel PH, Richard EA. Comparison of two cytological methods for detecting pulmonary haemorrhage in horses.. Vet Rec 2015 Sep 26;177(12):305.
    pubmed: 26374784doi: 10.1136/vr.103332google scholar: lookup
  13. Depecker M, Richard EA, Pitel PH, Fortier G, Leleu C, Couroucé-Malblanc A. Bronchoalveolar lavage fluid in Standardbred racehorses: influence of unilateral/bilateral profiles and cut-off values on lower airway disease diagnosis.. Vet J 2014 Jan;199(1):150-6.
    pubmed: 24225534doi: 10.1016/j.tvjl.2013.10.013google scholar: lookup
  14. Doucet MY, Viel L. Alveolar macrophage graded hemosiderin score from bronchoalveolar lavage in horses with exercise-induced pulmonary hemorrhage and controls.. J Vet Intern Med 2002 May-Jun;16(3):281-6.
    pubmed: 12041658doi: 10.1892/0891-6640(2002)016<0281:amghsf>2.3.co;2google scholar: lookup
  15. Friedrichs KR, Harr KE, Freeman KP, Szladovits B, Walton RM, Barnhart KF, Blanco-Chavez J. ASVCP reference interval guidelines: determination of de novo reference intervals in veterinary species and other related topics.. Vet Clin Pathol 2012 Dec;41(4):441-53.
    pubmed: 23240820doi: 10.1111/vcp.12006google scholar: lookup
  16. Hinchcliff KW, Couetil LL, Knight PK, Morley PS, Robinson NE, Sweeney CR, van Erck E. Exercise induced pulmonary hemorrhage in horses: American College of Veterinary Internal Medicine consensus statement.. J Vet Intern Med 2015 May-Jun;29(3):743-58.
    pmc: PMC4895427pubmed: 25996660doi: 10.1111/jvim.12593google scholar: lookup
  17. Hinchcliff KW, Jackson MA, Morley PS, Brown JA, Dredge AE, O'Callaghan PA, McCaffrey JP, Slocombe RE, Clarke AE. Association between exercise-induced pulmonary hemorrhage and performance in Thoroughbred racehorses.. J Am Vet Med Assoc 2005 Sep 1;227(5):768-74.
    pubmed: 16178399doi: 10.2460/javma.2005.227.768google scholar: lookup
  18. Hooi KS, Defarges AM, Jelovcic SV, Bienzle D. Bronchoalveolar lavage hemosiderosis in dogs and cats with respiratory disease.. Vet Clin Pathol 2019 Mar;48(1):42-49.
    pubmed: 30657606doi: 10.1111/vcp.12698google scholar: lookup
  19. Lopez Sanchez CM, Kogan C, Gold JR, Sellon DC, Bayly WM. Relationship between tracheobronchoscopic score and bronchoalveolar lavage red blood cell numbers in the diagnosis of exercise-induced pulmonary hemorrhage in horses.. J Vet Intern Med 2020 Jan;34(1):322-329.
    pmc: PMC6979084pubmed: 31880352doi: 10.1111/jvim.15676google scholar: lookup
  20. Marzahl C, Aubreville M, Bertram CA, Stayt J, Jasensky AK, Bartenschlager F, Fragoso-Garcia M, Barton AK, Elsemann S, Jabari S, Krauth J, Madhu P, Voigt J, Hill J, Klopfleisch R, Maier A. Deep Learning-Based Quantification of Pulmonary Hemosiderophages in Cytology Slides.. Sci Rep 2020 Aug 3;10(1):9795.
    pmc: PMC7398908pubmed: 32747665doi: 10.1038/s41598-020-65958-2google scholar: lookup
  21. Marzahl C, Aubreville M, Bertram CA, Maier J, Bergler C, Kröger C, Voigt J, Breininger K, Klopfleisch R, Maier A. EXACT: a collaboration toolset for algorithm-aided annotation of images with annotation version control.. Sci Rep 2021 Feb 23;11(1):4343.
    pmc: PMC7902667pubmed: 33623058doi: 10.1038/s41598-021-83827-4google scholar: lookup
  22. Marzahl C, Bertram CA, Aubreville M. Are fast labeling methods reliable? A case study of computer-aided expert annotations on microscopy slides. International Conference on Medical Image Computing and Computer-Assisted Intervention 2020:24–32.
  23. Marzahl C, Hill J, Stayt J, Bienzle D, Welker L, Wilm F, Voigt J, Aubreville M, Maier A, Klopfleisch R, Breininger K, Bertram CA. Inter-species cell detection - datasets on pulmonary hemosiderophages in equine, human and feline specimens.. Sci Data 2022 Jun 3;9(1):269.
    pmc: PMC9166691pubmed: 35660753doi: 10.1038/s41597-022-01389-0google scholar: lookup
  24. McAlpine ED, Michelow P. The cytopathologist's role in developing and evaluating artificial intelligence in cytopathology practice.. Cytopathology 2020 Sep;31(5):385-392.
    pubmed: 31957101doi: 10.1111/cyt.12799google scholar: lookup
  25. McKane SA, Slocombe RF. Sequential changes in bronchoalveolar cytology after autologous blood inoculation.. Equine Vet J Suppl 1999 Jul;(30):126-30.
    pubmed: 10659236doi: 10.1111/j.2042-3306.1999.tb05202.xgoogle scholar: lookup
  26. Meguro R, Asano Y, Odagiri S, Li C, Iwatsuki H, Shoumura K. Nonheme-iron histochemistry for light and electron microscopy: a historical, theoretical and technical review.. Arch Histol Cytol 2007 Apr;70(1):1-19.
    pubmed: 17558140doi: 10.1679/aohc.70.1google scholar: lookup
  27. Meguro R, Asano Y, Odagiri S, Li C, Iwatsuki H, Shoumura K. The presence of ferric and ferrous iron in the nonheme iron store of resident macrophages in different tissues and organs: histochemical demonstrations by the perfusion-Perls and -Turnbull methods in the rat.. Arch Histol Cytol 2005 Sep;68(3):171-83.
    pubmed: 16276023doi: 10.1679/aohc.68.171google scholar: lookup
  28. Meuten DJ, Moore FM, Donovan TA, Bertram CA, Klopfleisch R, Foster RA, Smedley RC, Dark MJ, Milovancev M, Stromberg P, Williams BH, Aubreville M, Avallone G, Bolfa P, Cullen J, Dennis MM, Goldschmidt M, Luong R, Miller AD, Miller MA, Munday JS, Roccabianca P, Salas EN, Schulman FY, Laufer-Amorim R, Asakawa MG, Craig L, Dervisis N, Esplin DG, George JW, Hauck M, Kagawa Y, Kiupel M, Linder K, Meichner K, Marconato L, Oblak ML, Santos RL, Simpson RM, Tvedten H, Whitley D. International Guidelines for Veterinary Tumor Pathology: A Call to Action.. Vet Pathol 2021 Sep;58(5):766-794.
    pubmed: 34282984doi: 10.1177/03009858211013712google scholar: lookup
  29. Meyer TS, Fedde MR, Gaughan EM, Langsetmo I, Erickson HH. Quantification of exercise-induced pulmonary haemorrhage with bronchoalveolar lavage.. Equine Vet J 1998 Jul;30(4):284-8.
    pubmed: 9705109doi: 10.1111/j.2042-3306.1998.tb04098.xgoogle scholar: lookup
  30. Newton JR, Wood JL. Evidence of an association between inflammatory airway disease and EIPH in young Thoroughbreds during training.. Equine Vet J Suppl 2002 Sep;(34):417-24.
    pubmed: 12405727doi: 10.1111/j.2042-3306.2002.tb05459.xgoogle scholar: lookup
  31. Perez-Moreno CI, Couëtil LL, Pratt SM, Ochoa-Acuña HG, Raskin RE, Russell MA. Effect of furosemide and furosemide-carbazochrome combination on exercise-induced pulmonary hemorrhage in Standardbred racehorses.. Can Vet J 2009 Aug;50(8):821-7.
    pmc: PMC2711466pubmed: 19881919
  32. Riedelsheimer B, Büchl-Zimmermann S. Färbungen. Romeis Mikroskopische Technik 2015:171–282.
  33. Step DL, Freeman KP, Gleed RD. Cytologic and endoscopic findings after intrapulmonary blood inoculation in horses. J Equine Vet Sci 1991;11:340–345.
  34. Sullivan S, Hinchcliff K. Update on exercise-induced pulmonary hemorrhage.. Vet Clin North Am Equine Pract 2015 Apr;31(1):187-98.
    pubmed: 25770069doi: 10.1016/j.cveq.2014.11.011google scholar: lookup
  35. Van Erck-Westergren E, Franklin SH, Bayly WM. Respiratory diseases and their effects on respiratory function and exercise capacity.. Equine Vet J 2013 May;45(3):376-87.
    pubmed: 23368813doi: 10.1111/evj.12028google scholar: lookup
  36. Zuraw A, Aeffner F. Whole-slide imaging, tissue image analysis, and artificial intelligence in veterinary pathology: An updated introduction and review.. Vet Pathol 2022 Jan;59(1):6-25.
    pubmed: 34521285doi: 10.1177/03009858211040484google scholar: lookup

Citations

This article has been cited 1 times.
  1. Mahalingam-Dhingra A, Bedenice D, Mazan MR. Bronchoalveolar lavage hemosiderosis in lightly active or sedentary horses.. J Vet Intern Med 2023 May-Jun;37(3):1243-1249.
    doi: 10.1111/jvim.16692pubmed: 36975043google scholar: lookup
Subscribe to our newsletter
Join 99,034 horse owners like you who receive our email updates on horse health and nutrition.