FREE SHIPPING over $40
OVER 9000 FIVE-STAR REVIEWS
5% OFF ALL SUBSCRIPTIONS
100% HAPPINESS GUARANTEE
Analyze Diet
0
Home / Equine Research Bank / Equine Vet J / Reliability of three scoring systems for assessing quality o...
Equine veterinary journal2025; doi: 10.1111/evj.70103

Reliability of three scoring systems for assessing quality of anaesthetic induction in horses.

Abstract: Several induction quality scoring systems (IQSS) have been described to evaluate drugs and risk factors of this anaesthetic period in horses, but no attempts to compare their reliability have been performed. Objective: To elucidate the reliability of three IQSS: the visual analogue scale (VAS), a simple descriptive scale (SDS), and a composite grading scale (CGS) proposed by the authors. Methods: Reliability study. Methods: Eight randomly selected video-recorded anaesthetic inductions from horses that underwent general anaesthesia were evaluated twice by four blinded evaluators with experience in equine anaesthesia, with a 1-month interval between assessments using the three aforementioned IQSS. A total of 64 evaluations per scale were generated. To assess reliability, intra- and inter-rater intraclass correlation coefficient (ICC), and their 95% confidence intervals (CI) were calculated based on a mean rating (k = 4), absolute agreement, 2-way random-effects model. Results: The inter-rater agreement was classified as moderate to good inter-rater reliability for all the scales, with the highest ICC found for the VAS (0.74 ± 0.11), followed by the CGS and the SDS (0.65 ± 0.22 and 0.63 ± 0.21, respectively). Intra-rater agreement results demonstrated very good reliability for both VAS and SDS (0.82 ± 0.08; 0.81 ± 0.18, respectively) and excellent reliability for the CGS (0.91 ± 0.08). Conclusions: The use of video-recordings instead of in situ evaluations, as the absence of audio may affect the assessment. Additionally, these findings are applicable only when free inductions are evaluated. Conclusions: The VAS and the novel CGS are reliable IQSS in horses, as are the widely used SDS. As the SDS are inconsistent across the literature, the VAS would be advised if multiple evaluators assess induction quality for research purposes, whereas the CGS would be selected for studies involving a single observer. We suggest routine inclusion of the VAS in the evaluation of the anaesthetic induction in horses.
© 2025 The Author(s). Equine Veterinary Journal published by John Wiley & Sons Ltd on behalf of EVJ Ltd.
Get Full Text
Publication Date: 2025-10-16 PubMed ID: 41099385DOI: 10.1111/evj.70103Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
LinkedInCopy
  • Journal Article

Summary

This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.

Overview

  • This study evaluates and compares the reliability of three different scoring systems used to assess the quality of anesthetic induction in horses.
  • The three scoring systems examined are the Visual Analogue Scale (VAS), Simple Descriptive Scale (SDS), and a newly proposed Composite Grading Scale (CGS).

Background and Purpose

  • Induction quality scoring systems (IQSS) help veterinarians objectively assess how well anesthesia is induced in horses, influencing drug choice and management to reduce risks.
  • Despite several existing IQSS, no prior research has compared their reliability directly, which is crucial for standardizing evaluations in research and clinical practice.
  • The study aims to fill this gap by comparing intra-rater (same evaluator over time) and inter-rater (different evaluators) reliability of the three scales.

Methods

  • Eight video-recorded anesthetic inductions of horses undergoing general anesthesia were randomly selected for assessment.
  • Four experienced equine anesthesia evaluators, blinded to each other’s ratings, scored these videos twice with a 1-month interval between sessions.
  • Each evaluator applied all three IQSS (VAS, SDS, CGS) to every video, generating 64 evaluations per scale (8 videos × 4 evaluators × 2 sessions).
  • Reliability was quantified using Intraclass Correlation Coefficient (ICC), which measures consistency/agreement across ratings:
    • Inter-rater ICC assessed agreement between different evaluators.
    • Intra-rater ICC assessed consistency of each evaluator’s ratings over time.
    • 95% confidence intervals were calculated to show precision of the ICC estimates.
    • A two-way random-effects model with absolute agreement was used for the ICC calculation, appropriate when raters are randomly sampled and absolute score agreement matters.

Results

  • Inter-rater reliability (agreement between different evaluators):
    • VAS demonstrated the highest reliability with ICC = 0.74 ± 0.11, classified as moderate to good reliability.
    • CGS reliability was slightly lower at ICC = 0.65 ± 0.22.
    • SDS showed comparable inter-rater reliability to CGS with ICC = 0.63 ± 0.21.
  • Intra-rater reliability (consistency of same evaluator over time):
    • VAS and SDS showed very good intra-rater reliability with ICC values around 0.81–0.82.
    • The CGS demonstrated excellent intra-rater reliability with ICC = 0.91 ± 0.08, indicating highly consistent ratings by individual evaluators over time.

Conclusions and Implications

  • The study confirms that all three scoring systems have acceptable reliability, but their strengths vary according to application:
    • VAS: Most reliable across multiple evaluators; should be preferred in research where several observers assess induction quality.
    • CGS: Superior intra-rater consistency; recommended for studies or clinical settings with a single observer.
    • SDS: Widely used but shows variable reliability in literature; less consistent compared to VAS and CGS in this study.
  • Limitations include the use of video recordings without audio, which may omit auditory cues important for assessing induction quality, and the findings apply primarily to situations involving free inductions.
  • The authors advocate for routine inclusion of the VAS during evaluation of anesthetic induction in horses to enhance consistency and comparability in both clinical practice and research.

Summary

  • This study systematically compares three methods for assessing equine anesthetic induction quality, a key period influencing anesthesia safety.
  • Findings suggest VAS is most suitable for multi-rater studies, CGS for single-observer assessments, and that SDS, although common, may lack consistency.
  • By recommending specific tools based on context, the study helps improve standardization and reliability in veterinary anesthetic assessment.

Cite This Article

APA
Villalba-Díez M, Benavente-Sánchez L, Bustamante R, Santiago-Llorente I, Villalba-Orero M. (2025). Reliability of three scoring systems for assessing quality of anaesthetic induction in horses. Equine Vet J. https://doi.org/10.1111/evj.70103

Publication

Equine veterinary journal
ISSN: 2042-3306
NlmUniqueID: 0173320
Country: United States
Language: English

Researcher Affiliations

Villalba-Díez, Marta
  • Department of Animal Medicine and Surgery, Faculty of Veterinary Medicine, Universidad Complutense de Madrid, Madrid, Spain.
Benavente-Sánchez, Leire
  • Department of Animal Medicine and Surgery, Faculty of Veterinary Medicine, Universidad Complutense de Madrid, Madrid, Spain.
Bustamante, Rocío
  • Department of Animal Medicine and Surgery, Faculty of Veterinary Medicine, Universidad Complutense de Madrid, Madrid, Spain.
Santiago-Llorente, Isabel
  • Department of Animal Medicine and Surgery, Faculty of Veterinary Medicine, Universidad Complutense de Madrid, Madrid, Spain.
Villalba-Orero, María
  • Department of Animal Medicine and Surgery, Faculty of Veterinary Medicine, Universidad Complutense de Madrid, Madrid, Spain.

Grant Funding

  • FPU22/03984 / Ministerio de Ciencia, Innovación y Universidades
  • EM-0000000620 / Public-private agreement UCM-Boehringer Ingelheim

References

This article includes 42 references
  1. Brock N, Hildebrand SV. A comparison of xylazine‐diazepam‐ketamine and xylazine‐guaifenesin‐ketamine in equine anesthesia. Vet Surg. 1990;19:468–474. https://doi.org/10.1111/j.1532-950x.1990.tb01237.x
  2. Hubbell JAE, Muir WW, Gorenberg E, Hopster K. A review of equine anesthetic induction: are all equine anesthetic inductions ‘crash’ inductions? J Equine Vet Sci. 2024;139:105130. https://doi.org/10.1016/j.jevs.2024.105130
  3. Muir WW, Hubbell JAE. Equine anesthesia: monitoring and emergency therapy. 2nd ed. St. Louis: Saunders Elsevier; 2009.
  4. Santiago‐Llorente I. Variables anestésicas como predictores de mortalidad en caballos operados de cólico. 2016. https://docta.ucm.es/entities/publication/9ffda340-c5df-48c3-948f-02112878a9d1
  5. Casoni D, Spadavecchia C, Wampfler B, Thormann W, Levionnois OL. Clinical and pharmacokinetic evaluation of S‐ketamine for intravenous general anaesthesia in horses undergoing field castration. Acta Vet Scand. 2015;57:21. https://doi.org/10.1186/s13028-015-0112-4
  6. Gozalo‐Marcilla M, Bettschart‐Wolfensberger R, Johnston M, Taylor PM, Redondo JI. Data collection for the fourth multicentre confidential enquiry of perioperative fatalities (CEPEF4) study: new technology and preliminary results. Animals. 2021;11:https://doi.org/10.3390/ani11092549
  7. Corletto F, Raisis BAA, Brearley JC. Comparison of morphine and butorphanol as pre‐anaesthetic agents in combination with romifidine for field castration in ponies. Vet Anaesth Analg. 2005;32:16–22. https://doi.org/10.1111/j.1467-2995.2004.00184.x
  8. Alonso B, Carregaro A, Cuypers C, Michielsen A, Gasthuys F, Schauvliege S. Effects of detomidine or romifidine during maintenance and recovery from isoflurane anaesthesia in horses. Vet Anaesth Analg. 2022;49:624–633. https://doi.org/10.1016/j.vaa.2022.07.004
  9. Farmer E, Chase‐Topping M, Lawson H, Clutton RE. Factors affecting the perception of recovery quality in horses after anaesthesia. Equine Vet J. 2014;46:328–332. https://doi.org/10.1111/evj.12133
  10. Ferreira TH, Brosnan RJ, Shilo‐Benjamini Y, Moore SB, Hollingsworth SR. Effects of ketamine, propofol, or thiopental administration on intraocular pressure and qualities of induction of and recovery from anesthesia in horses. Am J Vet Res. 2013;74:1070–1077. https://doi.org/10.2460/ajvr.74.8.1070
  11. Hubbell JAE, Hinchcliff KW, Schmall M, Muir WW, Robertson JT, Sams RA. Anesthetic, cardiorespiratory, and metabolic effects of four intravenous anesthetic regimens induced in horses immediately after maximal exercise. Am J Vet Res. 2000;61:1545–1552. https://doi.org/10.2460/ajvr.2000.61.1545
  12. Larenza MP, Ringer SK, Kutter APN, Conrot A, Theurillat R, Kummer M, et al. Evaluation of anesthesia recovery quality after low‐dose racemic or S‐ketamine infusions during anesthesia with isoflurane in horses. Am J Vet Res. 2009;70:710–718. https://doi.org/10.2460/ajvr.70.6.710
  13. Allison A, Robinson R, Jolliffe C, Taylor PM. Evaluation of the use of midazolam as a co‐induction agent with ketamine for anaesthesia in sedated ponies undergoing field castration. Equine Vet J. 2018;50:321–326. https://doi.org/10.1111/evj.12759
  14. Balko JA, Fogle C, Stuska SJ, Fogle JE, Posner LP. Retrospective and prospective assessment of butorphanol, azaperone and medetomidine (BAM™) for immobilisation of feral horses (Equus ferus caballus). Equine Vet J. 2022;54:549–555. https://doi.org/10.1111/evj.13490
  15. Hector RC, Rezende ML, Mama KR, Hess AM. Recovery quality following a single post‐anaesthetic dose of dexmedetomidine or romifidine in sevoflurane anaesthetised horses. Equine Vet J. 2020;52:685–691. https://doi.org/10.1111/evj.13238
  16. Smith MC, Bass L, Damone J, Mama K, Rao S. Comparison of xylazine and detomidine in combination with midazolam/ketamine for field castration in quarter horses. Equine Vet J. 2020;52:516–521. https://doi.org/10.1111/evj.13232
  17. Klöppel H, Leece EA. Comparison of ketamine and alfaxalone for induction and maintenance of anaesthesia in ponies undergoing castration. Vet Anaesth Analg. 2011;38:37–43. https://doi.org/10.1111/j.1467-2995.2010.00584.x
  18. Mama KR, Steffey EP, Pascoe PJ. Evaluation of propofol as a general anesthetic for horses. Vet Surg. 1995;24:188–194. https://doi.org/10.1111/j.1532-950x.1995.tb01317.x
  19. Kälin I, Henze IS, Ringer SK, Torgerson PR, Bettschart‐Wolfensberger R. Comparison of recovery quality following medetomidine versus xylazine balanced isoflurane anaesthesia in horses: a retrospective analysis. Animals. 2021;11:2440. https://doi.org/10.3390/ani11082440
  20. Gozalo‐Marcilla M, Steblaj B, Schauvliege S, Duchateau L, Gasthuys F. Comparison of the influence of two different constant‐rate infusions (dexmedetomidine versus morphine) on anaesthetic requirements, cardiopulmonary function and recovery quality in isoflurane anaesthetized horses. Res Vet Sci. 2013;95:1186–1194. https://doi.org/10.1016/j.rvsc.2013.09.014
  21. Clark‐Price SC, Posner LP, Gleed RD. Recovery of horses from general anesthesia in a darkened or illuminated recovery stall. Vet Anaesth Analg. 2008;35:473–479. https://doi.org/10.1111/j.1467-2995.2008.00414.x
  22. Donaldson LL, Dunlop GS, Holland MS, Burton BA. The recovery of horses from inhalant anesthesia: a comparison of halothane and isoflurane. Vet Surg. 2000;29:92–101. https://doi.org/10.1111/j.1532-950x.2000.00092.x
  23. Clark‐Price SC, Lascola KM, Carter JE, Da Cunha AF, Donaldson LL, Doherty TJ, et al. Assessment of agreement among diplomates of the American College of Veterinary Anesthesia and Analgesia for scoring the recovery of horses from anesthesia by use of subjective grading scales and development of a system for evaluation of the recovery of horses from anesthesia by use of accelerometry. Am J Vet Res. 2017;78:668676. https://doi.org/10.2460/ajvr.78.6.668
  24. Suthers JM, Christley RM, Clutton RE. Quantitative and qualitative comparison of three scoring systems for assessing recovery quality after general anaesthesia in horses. Vet Anaesth Analg. 2011;38:352–362. https://doi.org/10.1111/j.1467-2995.2011.00629.x
  25. Posner LP, Kasten JI, Kata C. Propofol with ketamine following sedation with xylazine for routine induction of general anaesthesia in horses. Vet Rec. 2013;173:550. https://doi.org/10.1136/vr.101906
  26. Keates HL, van Eps AW, Pearson MR. Alfaxalone compared with ketamine for induction of anaesthesia in horses following xylazine and guaifenesin. Vet Anaesth Analg. 2012;39:591–598. https://doi.org/10.1111/j.1467-2995.2012.00756.x
  27. Jarrett MA, Bailey KM, Messenger KM, Prange T, Gaines B, Posner LP. Recovery of horses from general anesthesia after induction with propofol and ketamine versus midazolam and ketamine. J Am Vet Med Assoc. 2018;253:101–107. https://doi.org/10.2460/javma.253.1.101
  28. Brosnan RJ, Steffey EP, Escobar A, Palazoglu M, Fiehn O. Anesthetic induction with guaifenesin and propofol in adult horses. Am J Vet Res. 2011;72:1569–1575. https://doi.org/10.2460/ajvr.72.12.1569
  29. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15:155–163. https://doi.org/10.1016/j.jcm.2016.02.012
  30. Wolfe KL, Hofmeister EH, Clark‐Price SC, Reed R, Quandt J. Development of the Auburn Induction Scale for evaluating induction quality in dogs. Vet Anaesth Analg. 2022;49:608–614. https://doi.org/10.1016/j.vaa.2022.08.010
  31. Fleiss JL. The design and analysis of clinical experiments. Wiley; 1999. https://onlinelibrary.wiley.com/doi/book/10.1002/9781118032923
  32. Portney LG, Watkins MP. Foundations of clinical research: applications to practice. 3rd ed. Upper Saddle River, NJ: Pearson/Prentice Hall; 2015.
  33. van Loon JPAM, Van Dierendonck MC. Objective pain assessment in horses (2014–2018). Vet J. 2018;242:1–7. https://doi.org/10.1016/j.tvjl.2018.10.001
  34. van Loon JPAM, Van Dierendonck MC. Monitoring acute equine visceral pain with the equine Utrecht University scale for composite pain assessment (EQUUS‐COMPASS) and the equine Utrecht University scale for facial assessment of pain (EQUUS‐FAP): a scale‐construction study. Vet J. 2015;206:356–364. https://doi.org/10.1016/j.tvjl.2015.08.023
  35. de Grauw JC, van Loon JPAM. Systematic pain assessment in horses. Vet J. 2016;209:14–22. https://doi.org/10.1016/j.tvjl.2015.07.030
  36. Vettorato E, Chase‐Topping ME, Clutton RE. A comparison of four systems for scoring recovery quality after general anaesthesia in horses. Equine Vet J. 2010;42:400–406. https://doi.org/10.1111/j.2042-3306.2010.00093.x
  37. Mama KR, Steffey EP, Pascoe PJ. Evaluation of propofol for general anesthesia in premedicated horses. Am J Vet Res. 1996;57:512–516.
  38. Ashley FH, Waterman‐Pearson AE, Whay HR. Behavioural assessment of pain in horses and donkeys: application to clinical practice and future studies. Equine Vet J. 2010;37:565–575. https://doi.org/10.2746/042516405775314826
  39. Viñuela‐Fernández I, Jones E, Chase‐Topping ME, Price J. Comparison of subjective scoring systems used to evaluate equine laminitis. Vet J. 2011;188:171–177. https://doi.org/10.1016/j.tvjl.2010.05.011
  40. de Grauw J, van Loon T. Clinical effects of two doses of butorphanol with detomidine for intravenous premedication of healthy warmblood horses. Vet Anaesth Analg. 2020;47:681–685. https://doi.org/10.1016/j.vaa.2019.11.010
  41. Tokushige H, Araki M, Kusano K, Arima D, Ito H, Yamazaki Y, et al. A retrospective comparison of induction with thiopental/guaifenesin and propofol/ketamine in thoroughbred racehorses anesthetized with sevoflurane and medetomidine during arthroscopic surgery. J Equine Sci. 2019;30:25–31. https://doi.org/10.1294/jes.30.25
  42. Aoki M, Wakuno A, Kushiro A, Mae N, Kakizaki M, Nagata S, et al. Evaluation of total intravenous anesthesia with propofol‐guaifenesin‐medetomidine and alfaxalone‐guaifenesin‐medetomidine in Thoroughbred horses undergoing castration. J Vet Med Sci. 2017;79:2011–2018. https://doi.org/10.1292/jvms.16-0658

Citations

This article has been cited 0 times.
Subscribe to our newsletter
Join 99,002 horse owners like you who receive our email updates on horse health and nutrition.