English
French
Rater agreement of visual lameness assessment in horses during lungeing.
Abstract: Lungeing is an important part of lameness examinations as the circular path may accentuate low-grade lameness. Movement asymmetries related to the circular path, to compensatory movements and to pain make the lameness evaluation complex. Scientific studies have shown high inter-rater variation when assessing lameness during straight line movement. Objective: The aim was to estimate inter- and intra-rater agreement of equine veterinarians evaluating lameness from videos of sound and lame horses during lungeing and to investigate the influence of veterinarians' experience and the objective degree of movement asymmetry on rater agreement. Methods: Cross-sectional observational study. Methods: Video recordings and quantitative gait analysis with inertial sensors were performed in 23 riding horses of various breeds. The horses were examined at trot on a straight line and during lungeing on soft or hard surfaces in both directions. One video sequence was recorded per condition and the horses were classified as forelimb lame, hindlimb lame or sound from objective straight line symmetry measurements. Equine veterinarians (n = 86), including 43 with >5 years of orthopaedic experience, participated in a web-based survey and were asked to identify the lamest limb on 60 videos, including 10 repeats. The agreements between (inter-rater) and within (intra-rater) veterinarians were analysed with κ statistics (Fleiss, Cohen). Results: Inter-rater agreement κ was 0.31 (0.38/0.25 for experienced/less experienced) and higher for forelimb (0.33) than for hindlimb lameness (0.11) or soundness (0.08) evaluation. Median intra-rater agreement κ was 0.57. Conclusions: Inter-rater agreement was poor for less experienced raters, and for all raters when evaluating hindlimb lameness. Since identification of the lame limb/limbs is a prerequisite for successful diagnosis, treatment and recovery, the high inter-rater variation when evaluating lameness on the lunge is likely to influence the accuracy and repeatability of lameness examinations and, indirectly, the efficacy of treatment.
© 2015 The Authors. Equine Veterinary Journal published by John Wiley & Sons Ltd on behalf of EVJ Ltd.
Publication Date: 2015-02-02 PubMed ID: 25399722PubMed Central: PMC4964936DOI: 10.1111/evj.12385Google Scholar: Lookup The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
- Journal Article
- Research Support
- Non-U.S. Gov't
Summary
This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.
The research investigates the level of agreement between equine veterinarians when identifying lameness in horses using a lungeing method, and how this agreement is impacted by the veterinarians’ experience and the degree of movement asymmetry in the horses. Results show that agreement levels were generally low, particularly for less experienced veterinarians and when assessing hindlimb lameness.
Overview of the Study
- The researchers conducted a cross-sectional observational study to gauge the level of agreement between different veterinarians (inter-rater) and within the same veterinarian (intra-rater) when diagnosing lameness in horses using the lungeing method.
- Recorded videos and quantitative gait analysis of 23 different riding horses were utilized in the study. These evaluations were performed during trotting on a straight line and during lungeing on both soft and hard surfaces in various directions.
- The horses were then classified according to objective straight line symmetry measurements as forelimb lame, hindlimb lame or sound.
- A web-based survey was used to collate the judgements of 86 equine veterinarians, including 43 with more than five years of orthopaedic experience, who were asked to identify the lamest limb from the 60 included videos.
Key Findings
- Results revealed that the inter-rater agreement κ was 0.31, with a higher agreement rate observed amongst more experienced veterinarians and forelimb lameness evaluation.
- Notably, the inter-rater agreement was far lower for hindlimb lameness and soundness evaluation, with values of κ = 0.11 and κ = 0.08, respectively.
- The median intra-rater agreement κ evaluated was 0.57.
Conclusion and Implications
- The study concludes that inter-rater agreement is considerably low for less experienced raters and for all raters when evaluating hindlimb lameness.
- This low level of agreement is significant because accurate identification of the lame limb is crucial for effective diagnosis, treatment, and recovery.
- The researchers suggest that the observed high level of variation when evaluating lameness using the lungeing method could impact the accuracy and repeatability of such examinations, thus indirectly influencing the efficacy of treatment.
Cite This Article
APA
Hammarberg M, Egenvall A, Pfau T, Rhodin M.
(2015).
Rater agreement of visual lameness assessment in horses during lungeing.
Equine Vet J, 48(1), 78-82.
https://doi.org/10.1111/evj.12385 Publication
Researcher Affiliations
- Department of Clinical Sciences, Swedish University of Agricultural Sciences, Uppsala, Sweden.
- Department of Clinical Sciences, Swedish University of Agricultural Sciences, Uppsala, Sweden.
- Department of Veterinary Clinical Sciences, The Royal Veterinary College, University of London, Hatfield, UK.
- Department of Clinical Sciences, Swedish University of Agricultural Sciences, Uppsala, Sweden.
MeSH Terms
- Animals
- Biomechanical Phenomena
- Cross-Sectional Studies
- Female
- Horse Diseases / diagnosis
- Horses
- Lameness, Animal / diagnosis
- Male
- Observer Variation
- Veterinarians
References
This article includes 20 references
- Kaneene JB, Ross WA, Miller R. The Michigan equine monitoring system. II. Frequencies and impact of selected health problems.. Prev Vet Med 1997 Feb;29(4):277-92.
- Penell JC, Egenvall A, Bonnett BN, Olson P, Pringle J. Specific causes of morbidity among Swedish horses insured for veterinary care between 1997 and 2000.. Vet Rec 2005 Oct 15;157(16):470-7.
- Baxter, G.M. , Stashak, T.S. and Adams, O.R. (2011) Adamsu2019 Lameness in Horses, 6th edn., Baxter G.M., Wileyu2010Blackwell, Chichester: p 115.
- Ross, M.D.S. (2003) Diagnosis and Management of Lameness in the Horse, 1st edn., Dyson S. and Ross M.W., W.B. Saunders, St Louis: p 71.
- Starke SD, Willems E, May SA, Pfau T. Vertical head and trunk movement adaptations of sound horses trotting in a circle on a hard surface.. Vet J 2012 Jul;193(1):73-80.
- Rhodin M, Pfau T, Roepstorff L, Egenvall A. Effect of lungeing on head and pelvic movement asymmetry in horses with induced lameness.. Vet J 2013 Dec;198 Suppl 1:e39-45.
- Pfau T, Stubbs NC, Kaiser LJ, Brown LE, Clayton HM. Effect of trotting speed and circle radius on movement symmetry in horses during lunging on a soft surface.. Am J Vet Res 2012 Dec;73(12):1890-9.
- Starke SD, Raistrick KJ, May SA, Pfau T. The effect of trotting speed on the evaluation of subtle lameness in horses.. Vet J 2013 Aug;197(2):245-52.
- Fuller CJ, Bladon BM, Driver AJ, Barr AR. The intra- and inter-assessor reliability of measurement of functional outcome by lameness scoring in horses.. Vet J 2006 Mar;171(2):281-6.
- Keegan KG, Dent EV, Wilson DA, Janicek J, Kramer J, Lacarrubba A, Walsh DM, Cassells MW, Esther TM, Schiltz P, Frees KE, Wilhite CL, Clark JM, Pollitt CC, Shaw R, Norris T. Repeatability of subjective evaluation of lameness in horses.. Equine Vet J 2010 Mar;42(2):92-7.
- Weishaupt MA, Wiestner T, Hogg HP, Jordan P, Auer JA, Barrey E. Assessment of gait irregularities in the horse: eye vs. gait analysis.. Equine Vet J Suppl 2001 Apr;(33):135-40.
- Merkens HW, Schamhardt HC. Evaluation of equine locomotion during different degrees of experimentally induced lameness. I: Lameness model and quantification of ground reaction force patterns of the limbs.. Equine Vet J Suppl 1988 Sep;(6):99-106.
- Keegan KG, Kramer J, Yonezawa Y, Maki H, Pai PF, Dent EV, Kellerman TE, Wilson DA, Reed SK. Assessment of repeatability of a wireless, inertial sensor-based lameness evaluation system for horses.. Am J Vet Res 2011 Sep;72(9):1156-63.
- Keegan KG, MacAllister CG, Wilson DA, Gedon CA, Kramer J, Yonezawa Y, Maki H, Pai PF. Comparison of an inertial sensor system with a stationary force plate for evaluation of horses with bilateral forelimb lameness.. Am J Vet Res 2012 Mar;73(3):368-74.
- Keegan KG, Pai PF, Wilson DA, Smith BK. Signal decomposition method of evaluating head movement to measure induced forelimb lameness in horses trotting on a treadmill.. Equine Vet J 2001 Sep;33(5):446-51.
- McCracken MJ, Kramer J, Keegan KG, Lopes M, Wilson DA, Reed SK, LaCarrubba A, Rasch M. Comparison of an inertial sensor system of lameness quantification with subjective lameness evaluation.. Equine Vet J 2012 Nov;44(6):652-6.
- Buchner HH, Savelberg HH, Schamhardt HC, Barneveld A. Head and trunk movement adaptations in horses with experimentally induced fore- or hindlimb lameness.. Equine Vet J 1996 Jan;28(1):71-6.
- Keegan KG, Yonezawa Y, Pai PF, Wilson DA, Kramer J. Evaluation of a sensor-based system of motion analysis for detection and quantification of forelimb and hind limb lameness in horses.. Am J Vet Res 2004 May;65(5):665-70.
- Arkell M, Archer RM, Guitian FJ, May SA. Evidence of bias affecting the interpretation of the results of local anaesthetic nerve blocks when assessing lameness in horses.. Vet Rec 2006 Sep 9;159(11):346-9.
- Greve L, Dyson SJ. An investigation of the relationship between hindlimb lameness and saddle slip.. Equine Vet J 2013 Sep;45(5):570-7.
Citations
This article has been cited 26 times.- Crecan CM, Peu0219tean CP. Inertial Sensor Technologies-Their Role in Equine Gait Analysis, a Review.. Sensors (Basel) 2023 Jul 11;23(14).
- Leclercq A, Lundblad J, Persson-Sjodin E, Ask K, Zetterberg E, Hernlund E, Haubro Andersen P, Rhodin M. Perceived sidedness and correlation to vertical movement asymmetries in young warmblood horses.. PLoS One 2023;18(7):e0288043.
- Zetterberg E, Leclercq A, Persson-Sjodin E, Lundblad J, Haubro Andersen P, Hernlund E, Rhodin M. Prevalence of vertical movement asymmetries at trot in Standardbred and Swedish Warmblood foals.. PLoS One 2023;18(4):e0284105.
- Lawin FJ, Bystru00f6m A, Roepstorff C, Rhodin M, Almlu00f6f M, Silva M, Andersen PH, Kjellstru00f6m H, Hernlund E. Is Markerless More or Less? Comparing a Smartphone Computer Vision Method for Equine Lameness Assessment to Multi-Camera Motion Capture.. Animals (Basel) 2023 Jan 24;13(3).
- Timmerman I, Macaire C, Hanne-Poujade S, Bertoni L, Martin P, Marin F, Chateau H. A Pilot Study on the Inter-Operator Reproducibility of a Wireless Sensors-Based System for Quantifying Gait Asymmetries in Horses.. Sensors (Basel) 2022 Dec 6;22(23).
- Leclercq A, Bystru00f6m A, Su00f6derlind M, Persson E, Rhodin M, Engell MT, Hernlund E. Evaluation of feedback methods for improved detection of hindlimb lameness in horses among riding instructors and trainers.. Front Vet Sci 2022;9:992954.
- Bailey J, Redpath A, Hallowell G, Bowen M. An objective study into the effects of an incline on naturally occurring lameness in horses.. Vet Med Sci 2022 Nov;8(6):2390-2395.
- Pfau T, Bolt DM, Fiske-Jackson A, Gerdes C, Hoenecke K, Lynch L, Perrier M, Smith RKW. Linear Discriminant Analysis for Investigating Differences in Upper Body Movement Symmetry in Horses before/after Diagnostic Analgesia in Relation to Expert Judgement.. Animals (Basel) 2022 Mar 17;12(6).
- Hardeman AM, Egenvall A, Serra Braganu00e7a FM, Swagemakers JH, Koene MHW, Roepstorff L, van Weeren R, Bystru00f6m A. Visual lameness assessment in comparison to quantitative gait analysis data in horses.. Equine Vet J 2022 Nov;54(6):1076-1085.
- Raulic J, Leung VS, Doss GA, Graham JE, Keller KA, Mans C, Sadar MJ, Vergneau-Grosset C, Pang DS. Development and Testing of a Sedation Scale for Use in Rabbits (Oryctolagus cuniculus).. J Am Assoc Lab Anim Sci 2021 Sep 1;60(5):549-555.
- Pfau T, Persson-Sjodin E, Gardner H, Orssten O, Hernlund E, Rhodin M. Effect of Speed and Surface Type on Individual Rein and Combined Left-Right Circle Movement Asymmetry in Horses on the Lunge.. Front Vet Sci 2021;8:692031.
- Tijssen M, Serra Braganu03c2a FM, Ask K, Rhodin M, Andersen PH, Telezhenko E, Bergsten C, Nielen M, Hernlund E. Kinematic gait characteristics of straight line walk in clinically sound dairy cows.. PLoS One 2021;16(7):e0253479.
- Spoormakers TJP, Graat EAM, Serra Braganu00e7a FM, Weeren PRV, Brommer H. Rater agreement for assessment of equine back mobility at walk and trot compared to quantitative gait analysis.. PLoS One 2021;16(6):e0252536.
- Haussler KK. Pressure Algometry for the Detection of Mechanical Nociceptive Thresholds in Horses.. Animals (Basel) 2020 Nov 24;10(12).
- Lesimple C. Indicators of Horse Welfare: State-of-the-Art.. Animals (Basel) 2020 Feb 13;10(2).
- Persson-Sjodin E, Hernlund E, Pfau T, Haubro Andersen P, Holm Forsstru00f6m K, Rhodin M. Effect of meloxicam treatment on movement asymmetry in riding horses in training.. PLoS One 2019;14(8):e0221117.
- Ekstrand C, Bondesson U, Giving E, Hedeland M, Ingvast-Larsson C, Jacobsen S, Lu00f6fgren M, Moen L, Rhodin M, Saetra T, Ranheim B. Disposition and effect of intra-articularly administered dexamethasone on lipopolysaccharide induced equine synovitis.. Acta Vet Scand 2019 Jun 20;61(1):28.
- Rhodin M, Persson-Sjodin E, Egenvall A, Serra Braganu00e7a FM, Pfau T, Roepstorff L, Weishaupt MA, Thomsen MH, van Weeren PR, Hernlund E. Vertical movement symmetry of the withers in horses with induced forelimb and hindlimb lameness at trot.. Equine Vet J 2018 Nov;50(6):818-824.
- Bosch S, Serra Braganu00e7a F, Marin-Perianu M, Marin-Perianu R, van der Zwaag BJ, Voskamp J, Back W, van Weeren R, Havinga P. EquiMoves: A Wireless Networked Inertial Measurement System for Objective Examination of Horse Gait.. Sensors (Basel) 2018 Mar 13;18(3).
- Serra Braganu00e7a FM, Rhodin M, Wiestner T, Hernlund E, Pfau T, van Weeren PR, Weishaupt MA. Quantification of the effect of instrumentation error in objective gait assessment in the horse on hindlimb symmetry parameters.. Equine Vet J 2018 May;50(3):370-376.
- Meeremans P, Yochum N, Kochzius M, Ampe B, Tuyttens FAM, Uhlmann SS. Inter-rater reliability of categorical versus continuous scoring of fish vitality: Does it affect the utility of the reflex action mortality predictor (RAMP) approach?. PLoS One 2017;12(7):e0179092.
- Rhodin M, Egenvall A, Haubro Andersen P, Pfau T. Head and pelvic movement asymmetries at trot in riding horses in training and perceived as free from lameness by the owner.. PLoS One 2017;12(4):e0176253.
- Wagner MC, Hecker KG, Pang DSJ. Sedation levels in dogs: a validation study.. BMC Vet Res 2017 Apr 18;13(1):110.
- Braganu00e7a FM, Bosch S, Voskamp JP, Marin-Perianu M, Van der Zwaag BJ, Vernooij JCM, van Weeren PR, Back W. Validation of distal limb mounted inertial measurement unit sensors for stride detection in Warmblood horses at walk and trot.. Equine Vet J 2017 Jul;49(4):545-551.
- Mirza MH, Bommala P, Richbourg HA, Rademacher N, Kearney MT, Lopez MJ. Gait Changes Vary among Horses with Naturally Occurring Osteoarthritis Following Intra-articular Administration of Autologous Platelet-Rich Plasma.. Front Vet Sci 2016;3:29.
- Rhodin M, Roepstorff L, French A, Keegan KG, Pfau T, Egenvall A. Head and pelvic movement asymmetry during lungeing in horses with symmetrical movement on the straight.. Equine Vet J 2016 May;48(3):315-20.
Secure Payments
Satisfaction Guarantee
We offer a 60-day guarantee to ensure you are completely satisfied.
Fast & Free Shipping
We provide fast and free shipping on all orders over $40.