Agreement between subjective evaluations and a markerless AI-based gait analysis system during lungeing assessment in traditional racehorses.

Overview

• This study investigates how well subjective evaluations of racehorse gait during lungeing agree with objective measures obtained from a markerless AI-based gait analysis system.
• The research also explores the consistency of subjective evaluations by different observers and by the same observer over time, assessing the potential use of AI tools in equine lameness diagnosis.

Background

• In equine veterinary practice, lameness during lungeing (circling on a lead rope) is often assessed subjectively by clinicians watching the horse trot.
• Subjective assessments can vary considerably, especially for mild or complex gait asymmetries, leading to inconsistent diagnoses.
• Objective gait analysis systems using AI have been developed to provide consistent, quantifiable data without the need for physical markers on the horse.
• This study focuses on traditional racehorses, analyzing gait during routine pre-race inspections, to compare subjective and AI-based evaluations.

Objectives

• Evaluate the level of agreement between subjective gait assessments by experienced clinicians and an AI-based markerless gait analysis system (OAI-MS) during lungeing.
• Determine the intra-observer (same evaluator at different times) and inter-observer (between different evaluators) agreement of subjective gait evaluations.
• Assess the short-term repeatability of the AI-based gait analysis system.

Methods

• Twenty-four traditional racehorses were assessed trotting on a soft surface during routine inspections (T0).
• Two experienced clinicians independently scored each horse’s gait on both reins using a modified AAEP scale:
• Original scale: 0-5, with 0 = no lameness.
• Converted to a 3-level ordinal scale: 0 = sound, 1 = mild lameness, 2 = severe lameness for analysis.
• At the same time, gait data were captured using the OAI-MS system, which is markerless and AI-powered.
• A subset of 10 horses was re-evaluated 10 days later (T1) to test repeatability of the AI measurements.
• Video-based reassessments (T2) were used for assessing intra-observer reliability for subjective scoring.
• Statistical agreement analyses used weighted Cohen’s kappa for two raters and Fleiss’ kappa for multiple raters or repeated measures.
• Significance threshold was set at p < 0.05.

Results

• Inter-observer agreement (between the two clinicians) was low to fair, with kappa values ranging from -0.20 (worse than chance) to 0.36 (fair agreement).
• Agreement between subjective evaluations and the OAI-MS system ranged from slight (0.13) to moderate (0.47), suggesting only partial concordance.
• Intra-observer agreement (same clinician repeated scoring) was fair at approximately 0.22, indicating some inconsistency over time.
• OAI-MS showed moderate repeatability with a kappa of 0.43 over the 10-day retest period, supporting its measurement stability.
• Higher agreement levels were observed in the evaluation of forelimb lameness compared to hindlimb lameness.
• Most differences in scoring were of low magnitude and likely reflect subtle or borderline cases.

Conclusions

• Subjective gait assessments during lungeing in traditional racehorses have limited and variable reliability, especially between different clinicians or across time.
• The markerless AI-based gait analysis system demonstrates better repeatability and provides objective data that partially agree with clinical judgment.
• The AI tool may be particularly useful as a complementary aid in clinical decision-making when lameness is subtle (mild asymmetries) or when clinician disagreement exists.
• Such technology has potential to improve diagnostic consistency and accuracy, especially for complex cases that are challenging to assess subjectively.

Implications for Practice and Future Research

• Veterinarians may consider incorporating AI-based gait analysis systems to support, not replace, clinical evaluations during lameness assessments.
• Further research could explore refinements in AI models for greater sensitivity and specificity, especially for hindlimb lameness.
• Larger-scale studies including different horse populations, surfaces, and gait conditions could help validate and generalize findings.
• Development of training programs could improve subjective evaluation consistency among clinicians.