Does Thirty-Minute Standardised Training Improve the Inter-Observer Reliability of the Horse Grimace Scale (HGS)? A Case Study.
Abstract: The Horse Grimace Scale (HGS) is a facial-expression-based pain coding system that enables a range of acute painful conditions in horses to be effectively identified. Using valid assessment methods to identify pain in horses is of a clear importance; however, the reliability of the assessment is highly dependent on the assessors' ability to use it. Training of new assessors plays a critical role in underpinning reliability. The aim of the study was to evaluate whether a 30-minute standardised training program on HGS is effective at improving the agreement between observers with no horse experience and when compared to an HGS expert. Two hundred and six undergraduate students with no horse experience were recruited. Prior to any training, observers were asked to score 10 pictures of horse faces using the six Facial Action Units (FAUs) of the HGS. Then, an HGS expert provided a 30-minute face-to-face training session, including detailed descriptions and example pictures of each FAU. After training, observers scored 10 different pictures. Cohen's k coefficient was used to determine inter-observer reliability between each observer and the expert; a paired-sample t-test was conducted to determine differences in agreement pre- and post-training. Pre-training, Cohen's k ranged from 0.20 for tension above the eye area to 0.68 for stiffly backwards ears. Post-training, the reliability for stiffly backwards ears and orbital tightening significantly increased, reaching Cohen's k values of 0.90 and 0.91 respectively (paired-sample t-test; < 0.001). The results suggest that this 30-minute face-to-face training session was not sufficient to allow observers without horse experience to effectively apply HGS. However, this standardised training program could represent a starting point for a more comprehensive training program for those without horse experience in order to increase their reliably in applying HGS.
Publication Date: 2020-04-30 PubMed ID: 32365927PubMed Central: PMC7277819DOI: 10.3390/ani10050781Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
- Journal Article
Summary
This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.
This study investigated whether a brief, standardized training session could improve the ability of untrained observers to use the Horse Grimace Scale (HGS), a tool used to assess pain in horses based on facial expressions. The training did not allow the inexperienced observers to effectively apply the HGS, but it significantly enhanced their ability to recognize certain features and could serve as a foundation for more comprehensive training.
Objective and Importance of the Study
- The study aimed to evaluate the effectiveness of a 30-minute standardised training program on the Horse Grimace Scale (HGS). This scale is crucial in identifying pain in horses based on facial expressions, an important component of equine care.
- Pain identification in horses is dependent on the accuracy of the assessment method and assessor’s observational skills. Hence, the reliability of the HGS is heavily reliant on proper training of assessors.
- This research sought to ascertain if a brief training would improve the ability of amateur observers, specifically those with no horse experience, to identify the pain in horses accurately using HGS, and how their performance compared with an HGS expert.
Methodology
- Two hundred and six undergraduate students with no horse experience were recruited for the study. They were asked to score 10 images of horse faces using the HGS’s six Facial Action Units (FAUs) before any training.
- A HGS expert then conducted a 30-minute training session, wherein they provided detailed instructions and example images for each FAU.
- Following the training, the same observers were asked to score 10 different horse face pictures.
- The researchers used Cohen’s k coefficient to determine the inter-observer reliability between each observer and the expert, and a paired-sample t-test was conducted to ascertain the differences in agreement before and after the training.
Results
- The results showed that pre-training agreement ranged from 0.20 for tension above the eye area to 0.68 for stiffly backward ears.
- Post-training, there was a significant increase in the recognition of stiffly backwards ears and orbital tightening, reaching Cohen’s k values of 0.90 and 0.91 respectively.
- Despite this increase in certain features recognition, the 30-minute training was insufficient to enable observers without experience in handling horses to effectively apply the HGS.
Conclusions
- The outcomes suggest that while a 30-minute training session was not enough for the observers to accurately use the HGS, the training did significantly improve their ability to recognize certain facial expressions associated with pain in horses.
- The researchers propose this short, standardised training could serve as an initial step for a more comprehensive training program tailored for observers with no horse experience, thereby improving their reliability in applying the HGS.
Cite This Article
APA
Dai F, Leach M, MacRae AM, Minero M, Costa ED.
(2020).
Does Thirty-Minute Standardised Training Improve the Inter-Observer Reliability of the Horse Grimace Scale (HGS)? A Case Study.
Animals (Basel), 10(5), 781.
https://doi.org/10.3390/ani10050781 Publication
Researcher Affiliations
- Dipartimento di Medicina Veterinaria, Università degli Studi di Milano, via Celoria 10, 20133 Milano, Italy.
- School of Natural and Environmental Sciences Agriculture, Newcastle University, Agriculture Building, Newcastle Upon Tyne NE1 7RU, UK.
- Animal Welfare Program, University of British Columbia, 2357 Main Mall, Vancouver, BC V6T 1Z4, Canada.
- Dipartimento di Medicina Veterinaria, Università degli Studi di Milano, via Celoria 10, 20133 Milano, Italy.
- Dipartimento di Medicina Veterinaria, Università degli Studi di Milano, via Celoria 10, 20133 Milano, Italy.
Grant Funding
- G1100563/1 / National Centre for the Replacement, Refinement and Reduction of Animals in Research
Conflict of Interest Statement
The authors declare no conflicts of interest.
References
This article includes 24 references
- Ashley FH, Waterman-Pearson AE, Whay HR. Behavioural assessment of pain in horses and donkeys: application to clinical practice and future studies.. Equine Vet J 2005 Nov;37(6):565-75.
- Dalla Costa E, Pascuzzo R, Leach MC, Dai F, Lebelt D, Vantini S, Minero M. Can grimace scales estimate the pain status in horses and mice? A statistical approach to identify a classifier.. PLoS One 2018;13(8):e0200339.
- Martin P, Bateson P. Measuring Behaviour: An Introductory Guide. 3rd ed. Cambridge University Press; Cambridge, MA, USA: 2007.
- Leach MC, Coulter CA, Richardson CA, Flecknell PA. Are we looking in the wrong place? Implications for behavioural-based pain assessment in rabbits (Oryctolagus cuniculi) and beyond?. PLoS One 2011 Mar 15;6(3):e13347.
- Ijichi C, Collins LM, Elwood RW. Pain expression is linked to personality in horses. Appl. Anim. Behav. Sci. 2014;152:38–43.
- Kaufman A, Rosenthal R. Can you believe my eyes? The importance of interobserver reliability statistics in observations of animal behaviour. Anim. Behav. 2009;78:1487–1491.
- Ashley F, Waterman-Pearson A, Whay HR, Pearson R, Muir C, Farrow M. Behavioural assessment of pain in horses and donkeys: Application to clinical practice and future studies. Proceedings of the Fifth International Colloquium on Working Equines; Addis Ababa, Ethiopia. 30 October–2 November 2006; pp. 15–23.
- Karlsten R, Ström K, Gunningberg L. Improving assessment of postoperative pain in surgical wards by education and training.. Qual Saf Health Care 2005 Oct;14(5):332-5.
- Ferrell BR, Grant M, Ritchey KJ, Ropchan R, Rivera LM. The pain resource nurse training program: a unique approach to pain management.. J Pain Symptom Manage 1993 Nov;8(8):549-56.
- DuBois C. Welfare in the Canadian Equine Industry: Understanding Perceptions and Pilot Testing of an On-Farm Assessment Tool. Ph.D. Thesis. University of Guelph; Guelph, ON, Canada: 2017.
- Dalla Costa E, Stucke D, Dai F, Minero M, Leach MC, Lebelt D. Using the Horse Grimace Scale (HGS) to Assess Pain Associated with Acute Laminitis in Horses (Equus caballus).. Animals (Basel) 2016 Aug 3;6(8).
- Dalla Costa E, Bracci D, Dai F, Lebelt D, Minero M. Do Different Emotional States Affect the Horse Grimace Scale Score? A Pilot Study. J. Equine Vet. Sci. 2017;54:114–117.
- Dalla Costa E, Minero M, Lebelt D, Stucke D, Canali E, Leach MC. Development of the Horse Grimace Scale (HGS) as a pain assessment tool in horses undergoing routine castration.. PLoS One 2014;9(3):e92281.
- Williams AC. Facial expression of pain: an evolutionary account.. Behav Brain Sci 2002 Aug;25(4):439-55; discussion 455-88.
- Keating SC, Thomas AA, Flecknell PA, Leach MC. Evaluation of EMLA cream for preventing pain during tattooing of rabbits: changes in physiological, behavioural and facial expression responses.. PLoS One 2012;7(9):e44437.
- Vasseur E, Gibbons J, Rushen J, de Passillé AM. Development and implementation of a training program to ensure high repeatability of body condition scoring of dairy cows.. J Dairy Sci 2013 Jul;96(7):4725-37.
- Mullan S, Edwards SA, Butterworth A, Whay HR, Main DCJ. Inter-observer reliability testing of pig welfare outcome measures proposed for inclusion within farm assurance schemes.. Vet J 2011 Nov;190(2):e100-e109.
- McHugh ML. Interrater reliability: the kappa statistic.. Biochem Med (Zagreb) 2012;22(3):276-82.
- Dai F, Dalla Costa E, Minero M. Efficacy of a standardized training on horse welfare indicators: A preliminary study. Int. J. Health Anim. Sci. Food Saf. 2018;5.
- Rushen J, Butterworth A, Swanson JC. Animal behavior and well-being symposium: Farm animal welfare assurance: science and application.. J Anim Sci 2011 Apr;89(4):1219-28.
- Thomsen PT, Baadsgaard NP. Intra- and inter-observer agreement of a protocol for clinical examination of dairy cows.. Prev Vet Med 2006 Jul 17;75(1-2):133-9.
- . Image Quality Metrics. Encyclopedia of Imaging Science and Technology John Wiley & Sons, Inc.; Jan 15, 2002.
- Gibbons J, Vasseur E, Rushen J, De Passillé A. A training programme to ensure high repeatability of injury scoring of dairy cows. Anim. Welf. 2012;21:379–388.
- March S, Brinkmann J, Winckler C. Effect of training on the inter-observe reliabiliy of lameness scoring in diary cattle. Anim. Welf. 2007;16:131–133.
Citations
This article has been cited 14 times.Use Nutrition Calculator
Check if your horse's diet meets their nutrition requirements with our easy-to-use tool Check your horse's diet with our easy-to-use tool
Talk to a Nutritionist
Discuss your horse's feeding plan with our experts over a free phone consultation Discuss your horse's diet over a phone consultation
Submit Diet Evaluation
Get a customized feeding plan for your horse formulated by our equine nutritionists Get a custom feeding plan formulated by our nutritionists
