FREE SHIPPING over $40
OVER 10000 FIVE-STAR REVIEWS
5% OFF ALL SUBSCRIPTIONS
100% HAPPINESS GUARANTEE
Analyze Diet
0
Equine veterinary journal2025; doi: 10.1111/evj.70116

Agreement between veterinarians and three objective evaluation systems in naturally occurring equine lameness.

Abstract: Subjective lameness evaluations are critical components of equine musculoskeletal health assessments. Objective approaches can supplement diagnosis and may be preferred for specific cases and scientific purposes. Objective: Evaluate agreements between subjective evaluation of two veterinarians and standard clinical interpretations from outputs of an AI-based smartphone application (Sleip; AI-SPA), the inertial measurement unit (IMU) system Equinosis Q Lameness Locator (LL), and the IMU system Equisym (ES). Methods: In vivo experiment. Methods: Twenty-five research horses (10-30 years) were evaluated on a straight-line trot. Limbs were independently graded and converted to an ordinal scale that, for objective systems, were converted from system-specific data outputs. Default settings and outputs for AI-SPA and LL were utilised to grade lameness while a manual process was developed for the ES. Pairwise agreement was calculated via weighted Cohen's κ, and agreement across rater types was calculated via Gwet's Agreement Coefficient 2 (GA2). Results: Objective evaluator agreement (GA2 = 0.84, 95% Confidence Interval (CI): 0.77-0.91) was higher than subjective evaluator agreement (GA2 = 0.73, 95% CI: 0.63-0.83) across all limbs. For all five evaluators/systems, overall forelimb agreement (GA2 = 0.82, 95% CI: 0.69-0.95) was greater than overall hindlimb agreement (GA2 = 0.73, 95% CI: 0.59-0.87). Pairwise agreement scores between the objective systems were often higher than those involving veterinary evaluators. The ES system often produced the highest agreement when compared with each rater individually. Conclusions: Horses were evaluated on a straight line only. Lameness diagnosis was limited to visual observation. Outcomes for each horse's four limbs were considered independent measurements. Conclusions: This work highlights the utility of commercially available objective evaluation systems, including the more recent ES system. Hindlimb asymmetries had lower agreement regardless of evaluator type. Objective systems had higher agreements when compared with subjective straight-line veterinary examination. The ability to uniformly assess asymmetries may assist diagnosis when compared with subjective evaluation alone.
© 2025 EVJ Ltd.
Get Full Text
Publication Date: 2025-11-16 PubMed ID: 41243239DOI: 10.1111/evj.70116Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
LinkedInCopy
  • Journal Article

Summary

This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.

Overview

  • This study evaluates the level of agreement between traditional subjective evaluations by veterinarians and results from three objective lameness detection systems in horses.
  • It aims to determine how well these objective technologies align with professional visual assessments in identifying equine lameness.

Background

  • Lameness evaluation in horses is a key part of assessing their musculoskeletal health.
  • Typically, veterinarians perform subjective visual examinations during a horse’s movement, often on a straight line.
  • Objective systems use technology to measure movement asymmetries and may provide quantitative data that supplement or support subjective evaluations.
  • Understanding agreements between subjective and objective evaluations can inform clinical decisions and improve diagnostic accuracy.

Objective

  • Compare agreement levels between two veterinarians’ subjective lameness grades and three objective systems:
  • An AI-based smartphone application called Sleip (AI-SPA)
  • The Equinosis Q Lameness Locator (LL), an inertial measurement unit (IMU) system
  • The Equisym (ES) IMU system, with a specially devised manual interpretation process
  • Determine the consistency and reliability of these methods in grading natural equine lameness.

    • Methods

    • Subjects: 25 research horses aged between 10 and 30 years were studied.
    • Evaluation setting: Horses were trotted on a straight line to enable observational and instrumented assessment.
    • Measurements:
      • Each horse’s limbs were independently scored for lameness by each evaluator and system.
      • Scoring was transformed into a common ordinal scale for comparability across subjective and objective formats.
      • For AI-SPA and LL, default automated output settings were used; for ES, a manual scoring method was applied.
    • Agreement analysis:
      • Weighted Cohen’s κ was used for pairwise agreement between raters/systems.
      • Gwet’s Agreement Coefficient 2 (GA2), a robust statistic for categorical agreement, was calculated for comparisons across all raters.

    Key Results

    • Overall, objective systems showed higher agreement among themselves (GA2 = 0.84) than between veterinarians (GA2 = 0.73).
    • Agreement was better when evaluating forelimbs (GA2 = 0.82) compared to hindlimbs (GA2 = 0.73) across all methods.
    • Pairwise agreements between objective systems (AI-SPA, LL, ES) were usually stronger than those involving veterinarians’ subjective evaluations.
    • The Equisym (ES) system tended to have the highest agreement with individual raters, suggesting strong compatibility with subjective assessments.

    Conclusions and Implications

    • The study highlights that objective measurement tools can reliably complement traditional veterinary assessments, particularly in recognizing asymmetries linked to lameness.
    • Lower agreement for hindlimb lameness indicates this area is more challenging to assess reliably, whether subjectively or objectively.
    • Higher objectivity and repeatability found in these systems could improve diagnostic consistency and reduce observer bias.
    • All evaluations were performed on horses moving on a straight line with only visual observation, which is a limitation—turning or other movements might influence results.
    • Each limb was considered independently, which is important because lameness in one limb can affect gait patterns in others.
    • The manual process developed for the ES system is promising and represents an advancement in objective diagnostic tools.
    • In clinical practice, integrating such objective technologies could enhance the accuracy and confidence of lameness diagnosis and treatment planning.

    Cite This Article

    APA
    McPeek JL, Menarim B, Sponseller B, McClendon M, Adam EN, Adams AA, Slone S, Page AE. (2025). Agreement between veterinarians and three objective evaluation systems in naturally occurring equine lameness. Equine Vet J. https://doi.org/10.1111/evj.70116

    Publication

    Equine veterinary journal
    ISSN: 2042-3306
    NlmUniqueID: 0173320
    Country: United States
    Language: English

    Researcher Affiliations

    McPeek, Jenna L
    • Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
    Menarim, Bruno
    • Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
    Sponseller, Beatrice
    • Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
    McClendon, Margaret
    • Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
    Adam, Emma N
    • Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
    Adams, Amanda A
    • Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
    Slone, Stacy
    • Department of Statistics, University of Kentucky, Lexington, Kentucky, USA.
    Page, Allen E
    • Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.

    Grant Funding

    • 58-5042-1-003 / U.S. Department of Agriculture

    References

    This article includes 32 references
    1. Egenvall A, Tranquille CA, Lönnell AC, Bitschnau C, Oomen A, Hernlund E. Days‐lost to training and competition in relation to workload in 263 elite show‐jumping horses in four European countries. Prev Vet Med 2013;112(3):387–400.
      doi: 10.1016/j.prevetmed.2013.09.013google scholar: lookup
    2. Dyson PK, Jackson BF, Pfeiffer DU, Price JS. Days lost from training by two‐ and three‐year‐old Thoroughbred horses: a survey of seven UK training yards. Equine Vet J 2008;40(7):650–657.
      doi: 10.2746/042516408x363242google scholar: lookup
    3. Jeffcott LB, Rossdale PD, Freestone J, Frank CJ, Towers‐Clark PF. An assessment of wastage in Thoroughbred racing from conception to 4 years of age. Equine Vet J 1982;14(3):185–198.
      doi: 10.1111/j.2042-3306.1982.tb02389.xgoogle scholar: lookup
    4. Keegan KG, Dent EV, Wilson DA, Janicek J, Kramer J, Lacarrubba A. Repeatability of subjective evaluation of lameness in horses. Equine Vet J 2010;42(2):92–97.
      doi: 10.2746/042516409x479568google scholar: lookup
    5. Keegan KG, Wilson DA, Wilson DJ, Smith B, Gaughan EM, Pleasant RS. Evaluation of mild lameness in horses trotting on a treadmill by clinicians and interns or residents and correlation of their assessments with kinematic gait analysis. Am J Vet Res 1998;59(11):1370–1377.
    6. Hammarberg M, Egenvall A, Pfau T, Rhodin M. Rater agreement of visual lameness assessment in horses during lungeing. Equine Vet J 2016;48(1):78–82.
      doi: 10.1111/evj.12385google scholar: lookup
    7. Poizat E, Gérard M, Macaire C, De Azevedo E, Denoix J‐M, Coudry V. Discrimination of the lame limb in horses using a machine learning method (support vector machine) based on asymmetry indices measured by the EQUISYM system. Sensors 2025;25(4):1095.
      doi: 10.3390/s25041095google scholar: lookup
    8. Leelamankong P, Estrada R, Mählmann K, Rungsri P, Lischer C. Agreement among equine veterinarians and between equine veterinarians and inertial sensor system during clinical examination of hindlimb lameness in horses. Equine Vet J 2020;52(2):326–331.
      doi: 10.1111/evj.13144google scholar: lookup
    9. Hardeman AM, Egenvall A, Serra Bragança FM, Swagemakers JH, Koene MHW, Roepstorff L. Visual lameness assessment in comparison to quantitative gait analysis data in horses. Equine Vet J 2022;54(6):1076–1085.
      doi: 10.1111/evj.13545google scholar: lookup
    10. Calle‐González N, Lo Feudo CM, Ferrucci F, Requena F, Stucchi L, Muñoz A. Objective assessment of equine locomotor symmetry using an inertial sensor system and artificial intelligence: a comparative study. Animals 2024;14(6):921.
      doi: 10.3390/ani14060921google scholar: lookup
    11. Keegan KG, MacAllister CG, Wilson DA, Gedon CA, Kramer J, Yonezawa Y. Comparison of an inertial sensor system with a stationary force plate for evaluation of horses with bilateral forelimb lameness. Am J Vet Res 2012;73(3):368–374.
      doi: 10.2460/ajvr.73.3.368google scholar: lookup
    12. Peloso JG, Stick JA, Soutas‐Little RW, Caron JC, DeCamp CE, Leach DH. Computer‐assisted three‐dimensional gait analysis of amphotericin‐induced carpal lameness in horses. Am J Vet Res 1993;54(9):1535–1543.
    13. Ishihara A, Bertone AL, Rajala‐Schultz PJ. Association between subjective lameness grade and kinetic gait parameters in horses with experimentally induced forelimb lameness. Am J Vet Res 2005;66(10):1805–1815.
      doi: 10.2460/ajvr.2005.66.1805google scholar: lookup
    14. Kai M, Aoki O, Hiraga A, Oki H, Tokuriki M. Use of an instrument sandwiched between the hoof and shoe to measure vertical ground reaction forces and three‐dimensional acceleration at the walk, trot, and canter in horses. Am J Vet Res 2000;61(8):979–985.
      doi: 10.2460/ajvr.2000.61.979google scholar: lookup
    15. Munoz‐Nates F, Chateau H, Van Hamme A, Camus M, Pauchard M, Ravary‐Plumioen B. Accelerometric and dynamometric measurements of the impact shock of the equine forelimb and hindlimb at high speed trot on six different tracks – preliminary study in one horse. Comput Methods Biomech Biomed Engin 2015;18(suppl 1):2012–2013.
      doi: 10.1080/10255842.2015.1069601google scholar: lookup
    16. Weishaupt MA, Hogg HP, Wiestner T, Denoth J, Stüssi E, Auer JA. Instrumented treadmill for measuring vertical ground reaction forces in horses. Am J Vet Res 2002;63(4):520–527.
      doi: 10.2460/ajvr.2002.63.520google scholar: lookup
    17. Hardeman AM, Byström A, Roepstorff L, Swagemakers JH, van Weeren PR, Serra Bragança FM. Range of motion and between‐measurement variation of spinal kinematics in sound horses at trot on the straight line and on the lunge. PLoS One 2020;15(2):e0222822.
      doi: 10.1371/journal.pone.0222822google scholar: lookup
    18. Donnell JR, Frisbie DD, King MR, Goodrich LR, Haussler KK. Comparison of subjective lameness evaluation, force platforms and an inertial‐sensor system to identify mild lameness in an equine osteoarthritis model. Vet J 2015;206(2):136–142.
      doi: 10.1016/j.tvjl.2015.08.004google scholar: lookup
    19. Bosch S, Serra Bragança F, Marin‐Perianu M, Marin‐Perianu R, Van der Zwaag BJ, Voskamp J. EquiMoves: a wireless networked inertial measurement system for objective examination of horse gait. Sensors 2018;18(3):850.
      doi: 10.3390/s18030850google scholar: lookup
    20. McCracken MJ, Kramer J, Keegan KG, Lopes M, Wilson DA, Reed SK. Comparison of an inertial sensor system of lameness quantification with subjective lameness evaluation. Equine Vet J 2012;44(6):652–656.
      doi: 10.1111/j.2042-3306.2012.00571.xgoogle scholar: lookup
    21. Persson‐Sjodin E, Hernlund E, Pfau T, Andersen PH, Forsström KH, Byström A. Withers vertical movement symmetry is useful for locating the primary lame limb in naturally occurring lameness. Equine Vet J 2024;56(1):76–88.
      doi: 10.1111/evj.13947google scholar: lookup
    22. Rhodin M, Persson‐Sjodin E, Egenvall A, Serra Bragança FM, Pfau T, Roepstorff L. Vertical movement symmetry of the withers in horses with induced forelimb and hindlimb lameness at trot. Equine Vet J 2018;50(6):818–824.
      doi: 10.1111/evj.12844google scholar: lookup
    23. Macaire C, Hanne‐Poujade S, De Azevedo E, Denoix J‐M, Coudry V, Jacquet S. Investigation of thresholds for asymmetry indices to represent the visual assessment of single limb lameness by expert veterinarians on horses trotting in a straight line. Animals 2022;12(24):3498.
      doi: 10.3390/ani12243498google scholar: lookup
    24. Kallerud AS, Marques‐Smith P, Bendiksen HK, Fjordbakk CT. Objective movement asymmetry in horses is comparable between markerless technology and sensor‐based systems. Equine Vet J 2025;57(1):115–125.
      doi: 10.1111/evj.14089google scholar: lookup
    25. Pfau T, Landsbergen K, Davis BL, Kenny O, Kernot N, Rochard N. Comparing inertial measurement units to MARKERLESS video analysis for movement symmetry in quarter horses. Sensors 2023;23(20):8414.
      doi: 10.3390/s23208414google scholar: lookup
    26. Keegan KG, Kramer J, Yonezawa Y, Maki H, Pai PF, Dent EV. Assessment of repeatability of a wireless, inertial sensor–based lameness evaluation system for horses. Am J Vet Res 2011;72(9):1156–1163.
      doi: 10.2460/ajvr.72.9.1156google scholar: lookup
    27. Lawin FJ, Byström A, Roepstorff C, Rhodin M, Almlöf M, Silva M. Is markerless more or less? Comparing a smartphone computer vision method for equine lameness assessment to multi‐camera motion capture. Animals 2023;13(3):390.
      doi: 10.3390/ani13030390google scholar: lookup
    28. Walsh P, Thornton J, Asato J, Walker N, McCoy G, Baal J. Approaches to describing inter‐rater reliability of the overall clinical appearance of febrile infants and toddlers in the emergency department. PeerJ 2014;2:e651.
      doi: 10.7717/peerj.651google scholar: lookup
    29. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22(3):276–282.
    30. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159–174.
      doi: 10.2307/2529310google scholar: lookup
    31. Rhodin M, Smit IH, Persson‐Sjodin E, Pfau T, Gunnarsson V, Björnsdóttir S. Timing of vertical head, withers and pelvis movements relative to the footfalls in different equine gaits and breeds. Animals (Basel) 2022;12(21):3053.
      doi: 10.3390/ani12213053google scholar: lookup
    32. Parkes RSV, Weller R, Groth AM, May S, Pfau T. Evidence of the development of ‘domain‐restricted’ expertise in the recognition of asymmetric motion characteristics of hindlimb lameness in the horse. Equine Vet J 2009;41(2):112–117.
      doi: 10.2746/042516408x343000google scholar: lookup

    Citations

    This article has been cited 0 times.
    Subscribe to our newsletter
    Join 99,970 horse owners like you who receive our email updates on horse health and nutrition.