Agreement between veterinarians and three objective evaluation systems in naturally occurring equine lameness.
Abstract: Subjective lameness evaluations are critical components of equine musculoskeletal health assessments. Objective approaches can supplement diagnosis and may be preferred for specific cases and scientific purposes. Objective: Evaluate agreements between subjective evaluation of two veterinarians and standard clinical interpretations from outputs of an AI-based smartphone application (Sleip; AI-SPA), the inertial measurement unit (IMU) system Equinosis Q Lameness Locator (LL), and the IMU system Equisym (ES). Methods: In vivo experiment. Methods: Twenty-five research horses (10-30 years) were evaluated on a straight-line trot. Limbs were independently graded and converted to an ordinal scale that, for objective systems, were converted from system-specific data outputs. Default settings and outputs for AI-SPA and LL were utilised to grade lameness while a manual process was developed for the ES. Pairwise agreement was calculated via weighted Cohen's κ, and agreement across rater types was calculated via Gwet's Agreement Coefficient 2 (GA2). Results: Objective evaluator agreement (GA2 = 0.84, 95% Confidence Interval (CI): 0.77-0.91) was higher than subjective evaluator agreement (GA2 = 0.73, 95% CI: 0.63-0.83) across all limbs. For all five evaluators/systems, overall forelimb agreement (GA2 = 0.82, 95% CI: 0.69-0.95) was greater than overall hindlimb agreement (GA2 = 0.73, 95% CI: 0.59-0.87). Pairwise agreement scores between the objective systems were often higher than those involving veterinary evaluators. The ES system often produced the highest agreement when compared with each rater individually. Conclusions: Horses were evaluated on a straight line only. Lameness diagnosis was limited to visual observation. Outcomes for each horse's four limbs were considered independent measurements. Conclusions: This work highlights the utility of commercially available objective evaluation systems, including the more recent ES system. Hindlimb asymmetries had lower agreement regardless of evaluator type. Objective systems had higher agreements when compared with subjective straight-line veterinary examination. The ability to uniformly assess asymmetries may assist diagnosis when compared with subjective evaluation alone.
© 2025 EVJ Ltd.
Publication Date: 2025-11-16 PubMed ID: 41243239DOI: 10.1111/evj.70116Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
- Journal Article
Summary
This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.
Overview
- This study evaluates the level of agreement between traditional subjective evaluations by veterinarians and results from three objective lameness detection systems in horses.
- It aims to determine how well these objective technologies align with professional visual assessments in identifying equine lameness.
Background
- Lameness evaluation in horses is a key part of assessing their musculoskeletal health.
- Typically, veterinarians perform subjective visual examinations during a horse’s movement, often on a straight line.
- Objective systems use technology to measure movement asymmetries and may provide quantitative data that supplement or support subjective evaluations.
- Understanding agreements between subjective and objective evaluations can inform clinical decisions and improve diagnostic accuracy.
Objective
- Compare agreement levels between two veterinarians’ subjective lameness grades and three objective systems:
- An AI-based smartphone application called Sleip (AI-SPA)
- The Equinosis Q Lameness Locator (LL), an inertial measurement unit (IMU) system
- The Equisym (ES) IMU system, with a specially devised manual interpretation process
Methods
- Subjects: 25 research horses aged between 10 and 30 years were studied.
- Evaluation setting: Horses were trotted on a straight line to enable observational and instrumented assessment.
- Measurements:
- Each horse’s limbs were independently scored for lameness by each evaluator and system.
- Scoring was transformed into a common ordinal scale for comparability across subjective and objective formats.
- For AI-SPA and LL, default automated output settings were used; for ES, a manual scoring method was applied.
- Agreement analysis:
- Weighted Cohen’s κ was used for pairwise agreement between raters/systems.
- Gwet’s Agreement Coefficient 2 (GA2), a robust statistic for categorical agreement, was calculated for comparisons across all raters.
Key Results
- Overall, objective systems showed higher agreement among themselves (GA2 = 0.84) than between veterinarians (GA2 = 0.73).
- Agreement was better when evaluating forelimbs (GA2 = 0.82) compared to hindlimbs (GA2 = 0.73) across all methods.
- Pairwise agreements between objective systems (AI-SPA, LL, ES) were usually stronger than those involving veterinarians’ subjective evaluations.
- The Equisym (ES) system tended to have the highest agreement with individual raters, suggesting strong compatibility with subjective assessments.
Conclusions and Implications
- The study highlights that objective measurement tools can reliably complement traditional veterinary assessments, particularly in recognizing asymmetries linked to lameness.
- Lower agreement for hindlimb lameness indicates this area is more challenging to assess reliably, whether subjectively or objectively.
- Higher objectivity and repeatability found in these systems could improve diagnostic consistency and reduce observer bias.
- All evaluations were performed on horses moving on a straight line with only visual observation, which is a limitation—turning or other movements might influence results.
- Each limb was considered independently, which is important because lameness in one limb can affect gait patterns in others.
- The manual process developed for the ES system is promising and represents an advancement in objective diagnostic tools.
- In clinical practice, integrating such objective technologies could enhance the accuracy and confidence of lameness diagnosis and treatment planning.
Cite This Article
APA
McPeek JL, Menarim B, Sponseller B, McClendon M, Adam EN, Adams AA, Slone S, Page AE.
(2025).
Agreement between veterinarians and three objective evaluation systems in naturally occurring equine lameness.
Equine Vet J.
https://doi.org/10.1111/evj.70116 Publication
Researcher Affiliations
- Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
- Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
- Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
- Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
- Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
- Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
- Department of Statistics, University of Kentucky, Lexington, Kentucky, USA.
- Maxwell H. Gluck Equine Research Center, University of Kentucky, Lexington, Kentucky, USA.
Grant Funding
- 58-5042-1-003 / U.S. Department of Agriculture
References
This article includes 32 references
- Egenvall A, Tranquille CA, Lönnell AC, Bitschnau C, Oomen A, Hernlund E, et al. Days‐lost to training and competition in relation to workload in 263 elite show‐jumping horses in four European countries. Prev Vet Med. 2013;112(3):387–400. https://doi.org/10.1016/j.prevetmed.2013.09.013
- Dyson PK, Jackson BF, Pfeiffer DU, Price JS. Days lost from training by two‐ and three‐year‐old Thoroughbred horses: a survey of seven UK training yards. Equine Vet J. 2008;40(7):650–657. https://doi.org/10.2746/042516408X363242
- Jeffcott LB, Rossdale PD, Freestone J, Frank CJ, Towers‐Clark PF. An assessment of wastage in Thoroughbred racing from conception to 4 years of age. Equine Vet J. 1982;14(3):185–198. https://doi.org/10.1111/j.2042-3306.1982.tb02389.x
- Keegan KG, Dent EV, Wilson DA, Janicek J, Kramer J, Lacarrubba A, et al. Repeatability of subjective evaluation of lameness in horses. Equine Vet J. 2010;42(2):92–97. https://doi.org/10.2746/042516409X479568
- Keegan KG, Wilson DA, Wilson DJ, Smith B, Gaughan EM, Pleasant RS, et al. Evaluation of mild lameness in horses trotting on a treadmill by clinicians and interns or residents and correlation of their assessments with kinematic gait analysis. Am J Vet Res. 1998;59(11):1370–1377.
- Hammarberg M, Egenvall A, Pfau T, Rhodin M. Rater agreement of visual lameness assessment in horses during lungeing. Equine Vet J. 2016;48(1):78–82. https://doi.org/10.1111/evj.12385
- Poizat E, Gérard M, Macaire C, De Azevedo E, Denoix J‐M, Coudry V, et al. Discrimination of the lame limb in horses using a machine learning method (support vector machine) based on asymmetry indices measured by the EQUISYM system. Sensors. 2025;25(4):1095. https://doi.org/10.3390/s25041095
- Leelamankong P, Estrada R, Mählmann K, Rungsri P, Lischer C. Agreement among equine veterinarians and between equine veterinarians and inertial sensor system during clinical examination of hindlimb lameness in horses. Equine Vet J. 2020;52(2):326–331. https://doi.org/10.1111/evj.13144
- Hardeman AM, Egenvall A, Serra Bragança FM, Swagemakers JH, Koene MHW, Roepstorff L, et al. Visual lameness assessment in comparison to quantitative gait analysis data in horses. Equine Vet J. 2022;54(6):1076–1085. https://doi.org/10.1111/evj.13545
- Calle‐González N, Lo Feudo CM, Ferrucci F, Requena F, Stucchi L, Muñoz A. Objective assessment of equine locomotor symmetry using an inertial sensor system and artificial intelligence: a comparative study. Animals. 2024;14(6):921. https://doi.org/10.3390/ani14060921
- Keegan KG, MacAllister CG, Wilson DA, Gedon CA, Kramer J, Yonezawa Y, et al. Comparison of an inertial sensor system with a stationary force plate for evaluation of horses with bilateral forelimb lameness. Am J Vet Res. 2012;73(3):368–374. https://doi.org/10.2460/ajvr.73.3.368
- Peloso JG, Stick JA, Soutas‐Little RW, Caron JC, DeCamp CE, Leach DH. Computer‐assisted three‐dimensional gait analysis of amphotericin‐induced carpal lameness in horses. Am J Vet Res. 1993;54(9):1535–1543.
- Ishihara A, Bertone AL, Rajala‐Schultz PJ. Association between subjective lameness grade and kinetic gait parameters in horses with experimentally induced forelimb lameness. Am J Vet Res. 2005;66(10):1805–1815. https://doi.org/10.2460/ajvr.2005.66.1805
- Kai M, Aoki O, Hiraga A, Oki H, Tokuriki M. Use of an instrument sandwiched between the hoof and shoe to measure vertical ground reaction forces and three‐dimensional acceleration at the walk, trot, and canter in horses. Am J Vet Res. 2000;61(8):979–985. https://doi.org/10.2460/ajvr.2000.61.979
- Munoz‐Nates F, Chateau H, Van Hamme A, Camus M, Pauchard M, Ravary‐Plumioen B, et al. Accelerometric and dynamometric measurements of the impact shock of the equine forelimb and hindlimb at high speed trot on six different tracks – preliminary study in one horse. Comput Methods Biomech Biomed Engin. 2015;18(suppl 1):2012–2013. https://doi.org/10.1080/10255842.2015.1069601
- Weishaupt MA, Hogg HP, Wiestner T, Denoth J, Stüssi E, Auer JA. Instrumented treadmill for measuring vertical ground reaction forces in horses. Am J Vet Res. 2002;63(4):520–527. https://doi.org/10.2460/ajvr.2002.63.520
- Hardeman AM, Byström A, Roepstorff L, Swagemakers JH, van Weeren PR, Serra Bragança FM. Range of motion and between‐measurement variation of spinal kinematics in sound horses at trot on the straight line and on the lunge. PLoS One. 2020;15(2):e0222822. https://doi.org/10.1371/journal.pone.0222822
- Donnell JR, Frisbie DD, King MR, Goodrich LR, Haussler KK. Comparison of subjective lameness evaluation, force platforms and an inertial‐sensor system to identify mild lameness in an equine osteoarthritis model. Vet J. 2015;206(2):136–142. https://doi.org/10.1016/j.tvjl.2015.08.004
- Bosch S, Serra Bragança F, Marin‐Perianu M, Marin‐Perianu R, Van der Zwaag BJ, Voskamp J, et al. EquiMoves: a wireless networked inertial measurement system for objective examination of horse gait. Sensors. 2018;18(3):850. https://doi.org/10.3390/s18030850
- McCracken MJ, Kramer J, Keegan KG, Lopes M, Wilson DA, Reed SK, et al. Comparison of an inertial sensor system of lameness quantification with subjective lameness evaluation. Equine Vet J. 2012;44(6):652–656. https://doi.org/10.1111/j.2042-3306.2012.00571.x
- Persson‐Sjodin E, Hernlund E, Pfau T, Andersen PH, Forsström KH, Byström A, et al. Withers vertical movement symmetry is useful for locating the primary lame limb in naturally occurring lameness. Equine Vet J. 2024;56(1):76–88. https://doi.org/10.1111/evj.13947
- Rhodin M, Persson‐Sjodin E, Egenvall A, Serra Bragança FM, Pfau T, Roepstorff L, et al. Vertical movement symmetry of the withers in horses with induced forelimb and hindlimb lameness at trot. Equine Vet J. 2018;50(6):818–824. https://doi.org/10.1111/evj.12844
- Macaire C, Hanne‐Poujade S, De Azevedo E, Denoix J‐M, Coudry V, Jacquet S, et al. Investigation of thresholds for asymmetry indices to represent the visual assessment of single limb lameness by expert veterinarians on horses trotting in a straight line. Animals. 2022;12(24):3498. https://doi.org/10.3390/ani12243498
- Kallerud AS, Marques‐Smith P, Bendiksen HK, Fjordbakk CT. Objective movement asymmetry in horses is comparable between markerless technology and sensor‐based systems. Equine Vet J. 2025;57(1):115–125. https://doi.org/10.1111/evj.14089
- Pfau T, Landsbergen K, Davis BL, Kenny O, Kernot N, Rochard N, et al. Comparing inertial measurement units to MARKERLESS video analysis for movement symmetry in quarter horses. Sensors. 2023;23(20):8414. https://doi.org/10.3390/s23208414
- Keegan KG, Kramer J, Yonezawa Y, Maki H, Pai PF, Dent EV, et al. Assessment of repeatability of a wireless, inertial sensor–based lameness evaluation system for horses. Am J Vet Res. 2011;72(9):1156–1163. https://doi.org/10.2460/ajvr.72.9.1156
- Lawin FJ, Byström A, Roepstorff C, Rhodin M, Almlöf M, Silva M, et al. Is markerless more or less? Comparing a smartphone computer vision method for equine lameness assessment to multi‐camera motion capture. Animals. 2023;13(3):390. https://doi.org/10.3390/ani13030390
- Walsh P, Thornton J, Asato J, Walker N, McCoy G, Baal J, et al. Approaches to describing inter‐rater reliability of the overall clinical appearance of febrile infants and toddlers in the emergency department. PeerJ. 2014;2:e651. https://doi.org/10.7717/peerj.651
- McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276–282.
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. https://doi.org/10.2307/2529310
- Rhodin M, Smit IH, Persson‐Sjodin E, Pfau T, Gunnarsson V, Björnsdóttir S, et al. Timing of vertical head, withers and pelvis movements relative to the footfalls in different equine gaits and breeds. Animals (Basel). 2022;12(21):3053. https://doi.org/10.3390/ani12213053
- Parkes RSV, Weller R, Groth AM, May S, Pfau T. Evidence of the development of ‘domain‐restricted’ expertise in the recognition of asymmetric motion characteristics of hindlimb lameness in the horse. Equine Vet J. 2009;41(2):112–117. https://doi.org/10.2746/042516408X343000
Citations
This article has been cited 0 times.Use Nutrition Calculator
Check if your horse's diet meets their nutrition requirements with our easy-to-use tool Check your horse's diet with our easy-to-use tool
Talk to a Nutritionist
Discuss your horse's feeding plan with our experts over a free phone consultation Discuss your horse's diet over a phone consultation
Submit Diet Evaluation
Get a customized feeding plan for your horse formulated by our equine nutritionists Get a custom feeding plan formulated by our nutritionists
