FREE SHIPPING over $40
OVER 9000 FIVE-STAR REVIEWS
5% OFF ALL SUBSCRIPTIONS
100% HAPPINESS GUARANTEE
Analyze Diet
0
Home / Equine Research Bank / Vet Sci / Analysis of Various Facial Expressions of Horses as a Welfar...
Veterinary sciences2023; 10(4); 283; doi: 10.3390/vetsci10040283

Analysis of Various Facial Expressions of Horses as a Welfare Indicator Using Deep Learning.

Abstract: This study aimed to prove that deep learning can be effectively used for identifying various equine facial expressions as welfare indicators. In this study, a total of 749 horses (healthy: 586 and experiencing pain: 163) were investigated. Moreover, a model for recognizing facial expressions based on images and their classification into four categories, i.e., resting horses (RH), horses with pain (HP), horses immediately after exercise (HE), and horseshoeing horses (HH), was developed. The normalization of equine facial posture revealed that the profile (99.45%) had higher accuracy than the front (97.59%). The eyes-nose-ears detection model achieved an accuracy of 98.75% in training, 81.44% in validation, and 88.1% in testing, with an average accuracy of 89.43%. Overall, the average classification accuracy was high; however, the accuracy of pain classification was low. These results imply that various facial expressions in addition to pain may exist in horses depending on the situation, degree of pain, and type of pain experienced by horses. Furthermore, automatic pain and stress recognition would greatly enhance the identification of pain and other emotional states, thereby improving the quality of equine welfare.
Get Full Text
Publication Date: 2023-04-10 PubMed ID: 37104439PubMed Central: PMC10141195DOI: 10.3390/vetsci10040283Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
LinkedInCopy
  • Journal Article

Summary

This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.

The research article demonstrates the possibility of using deep learning to identify different facial expressions of horses, which can serve as indicators of their welfare.

Objective of the Study

The primary objective of this study was to prove the efficiency of deep learning in the identification of various facial expressions in horses which can serve to indicate their welfare.

  • Specifically, the study aimed to develop a deep learning model that could classify images of horses’ faces into four categories based on expression: resting horses (RH), horses with pain (HP), horses immediately after exercise (HE), and horseshoeing horses (HH).

Methodology

  • The study used a sample size of 749 horses, out of which 586 were healthy and 163 were experiencing pain.
  • Images of the horses’ faces were collected and used to train and validate the deep learning model.
  • The facial features (eyes, nose, and ears) were categorized and compared for accuracy across two different postures – profile and front.
  • The deep learning model was subsequently tested for accuracy in identifying the correct category of facial expression.

Results and Findings

  • The results demonstrated that the recognition of the profile posture had higher accuracy (99.45%) than the front (97.59%).
  • In the case of feature detection (eyes-nose-ears), the model exhibited an accuracy of 98.75% in training, 81.44% in validation, and 88.1% in testing. The average accuracy amounted to 89.43%.
  • While the overall average classification accuracy was high, the model’s accuracy was significantly lower when it came to classifying pain expressions. This suggests that in addition to pain, horses may exhibit various other facial expressions depending on their situation, the degree of pain, and the type of pain experienced.

Implications

  • This research implies that deep learning can be effectively used to recognize different facial expressions of horses which could significantly enhance automatic pain and stress recognition.
  • Not only would this improve our understanding of a horse’s emotional states, but it also has potential applications in enriching the quality of equine welfare by enabling quick and highly accurate detection of pain and stress.

Cite This Article

APA
Kim SM, Cho GJ. (2023). Analysis of Various Facial Expressions of Horses as a Welfare Indicator Using Deep Learning. Vet Sci, 10(4), 283. https://doi.org/10.3390/vetsci10040283

Publication

Veterinary sciences
ISSN: 2306-7381
NlmUniqueID: 101680127
Country: Switzerland
Language: English
Volume: 10
Issue: 4
PII: 283

Researcher Affiliations

Kim, Su Min
  • College of Veterinary Medicine, Kyungpook National University, Daegu 41566, Republic of Korea.
Cho, Gil Jae
  • College of Veterinary Medicine, Kyungpook National University, Daegu 41566, Republic of Korea.

Grant Funding

  • NRF-2020R1I1A3067905 / Ministry of Education

Conflict of Interest Statement

The authors declare no conflict of interest.

References

This article includes 67 references
  1. Darwin C, Prodger P. The Expression of the Emotions in Man and Animals. .
    doi: 10.1017/CBO9781139833813google scholar: lookup
  2. Leach MC, Klaus K, Miller AL, Scotto di Perrotolo M, Sotocinal SG, Flecknell PA. The assessment of post-vasectomy pain in mice using behaviour and the Mouse Grimace Scale.. PLoS One 2012;7(4):e35656.
    doi: 10.1371/journal.pone.0035656pmc: PMC3338444pubmed: 22558191google scholar: lookup
  3. Van Rysewyk S. Nonverbal indicators of pain. Anim. Sentience 2016;1:30.
    doi: 10.51291/2377-7478.1061google scholar: lookup
  4. Price J, Marques JM, Welsh EM, Waran NK. Pilot epidemiological study of attitudes towards pain in horses.. Vet Rec 2002 Nov 9;151(19):570-5.
    doi: 10.1136/vr.151.19.570pubmed: 12452357google scholar: lookup
  5. Morrel-Samuels P, Krauss R.M. Cartesian analysis: A computer-video interface for measuring motion without physical contact. Behav. Res. Methods Instrum. Comput. 1990;22:466–470.
    doi: 10.3758/BF03203196google scholar: lookup
  6. Pereira P, Oliveira R.F. A simple method using a single video camera to determine the three-dimensional position of a fish. Behav. Res. Meth. Instrum. Comput. 1994;26:443–446.
    doi: 10.3758/BF03204663google scholar: lookup
  7. Santucci A.C. An affordable computer-aided method for conducting Morris water maze testing. Behav. Res. Meth. Instrum. Comput. 1995;27:60–64.
    doi: 10.3758/BF03203622google scholar: lookup
  8. Gleerup K.B, Lindegaard C. Recognition and quantification of pain in horses: A tutorial review. Equine Vet. Educ. 2016;28:47–57.
    doi: 10.1111/eve.12383google scholar: lookup
  9. Graubner C, Gerber V, Doherr M, Spadavecchia C. Clinical application and reliability of a post abdominal surgery pain assessment scale (PASPAS) in horses.. Vet J 2011 May;188(2):178-83.
    doi: 10.1016/j.tvjl.2010.04.029pubmed: 20627635google scholar: lookup
  10. Price J, Catriona S, Welsh EM, Waran NK. Preliminary evaluation of a behaviour-based system for assessment of post-operative pain in horses following arthroscopic surgery.. Vet Anaesth Analg 2003 Jul;30(3):124-37.
    doi: 10.1046/j.1467-2995.2003.00139.xpubmed: 14498844google scholar: lookup
  11. Raekallio M, Taylor P.M, Bloomfield M.A. Comparison of methods for evaluation of pain and distress after orthopaedic surgery in horses. Vet. Anaesth. Analg. 1997;24:17–20.
    doi: 10.1111/j.1467-2995.1997.tb00150.xgoogle scholar: lookup
  12. Sellon DC, Roberts MC, Blikslager AT, Ulibarri C, Papich MG. Effects of continuous rate intravenous infusion of butorphanol on physiologic and outcome variables in horses after celiotomy.. J Vet Intern Med 2004 Jul-Aug;18(4):555-63.
    doi: 10.1111/j.1939-1676.2004.tb02585.xpubmed: 15320598google scholar: lookup
  13. van Loon JP, Van Dierendonck MC. Monitoring acute equine visceral pain with the Equine Utrecht University Scale for Composite Pain Assessment (EQUUS-COMPASS) and the Equine Utrecht University Scale for Facial Assessment of Pain (EQUUS-FAP): A scale-construction study.. Vet J 2015 Dec;206(3):356-64.
    doi: 10.1016/j.tvjl.2015.08.023pubmed: 26526526google scholar: lookup
  14. de Grauw JC, van Loon JP. Systematic pain assessment in horses.. Vet J 2016 Mar;209:14-22.
    doi: 10.1016/j.tvjl.2015.07.030pubmed: 26831169google scholar: lookup
  15. Love E.J. Assessment and management of pain in horses. Equine Vet. Educ. 2009;21:46–48.
    doi: 10.2746/095777309X390290google scholar: lookup
  16. Gleerup KB, Forkman B, Lindegaard C, Andersen PH. An equine pain face.. Vet Anaesth Analg 2015 Jan;42(1):103-14.
    doi: 10.1111/vaa.12212pmc: PMC4312484pubmed: 25082060google scholar: lookup
  17. Andersen PH, Broomé S, Rashid M, Lundblad J, Ask K, Li Z, Hernlund E, Rhodin M, Kjellström H. Towards Machine Recognition of Facial Expressions of Pain in Horses.. Animals (Basel) 2021 Jun 1;11(6).
    doi: 10.3390/ani11061643pmc: PMC8229776pubmed: 34206077google scholar: lookup
  18. Hummel H.I, Pessanha F, Salah A.A, van Loon T, Veltkamp R.C. Automatic pain detection on horse and donkey faces. Proceedings of the 15th IEEE International Conference on Automatic Face and Gesture Recognition; Buenos Aires, Argentina. 16–20 November 2020; pp. 793–800.
    doi: 10.1109/FG47880.2020.00114google scholar: lookup
  19. Williams AC. Facial expression of pain: an evolutionary account.. Behav Brain Sci 2002 Aug;25(4):439-55; discussion 455-88.
    doi: 10.1017/S0140525X02000080pubmed: 12879700google scholar: lookup
  20. Ashley FH, Waterman-Pearson AE, Whay HR. Behavioural assessment of pain in horses and donkeys: application to clinical practice and future studies.. Equine Vet J 2005 Nov;37(6):565-75.
    doi: 10.2746/042516405775314826pubmed: 16295937google scholar: lookup
  21. Coles B, Birgitsdottir L, Andersen P.H. Out of sight but not out of clinician’s mind: Using remote video surveillance to disclose concealed pain behavior in hospitalized horses. Proceedings of the International Association for the Study of Pain 17th World Congress; Boston, MA, USA. 12 September 2018; p. 471121.
  22. Torcivia C, McDonnell S. In-Person Caretaker Visits Disrupt Ongoing Discomfort Behavior in Hospitalized Equine Orthopedic Surgical Patients.. Animals (Basel) 2020 Jan 27;10(2).
    doi: 10.3390/ani10020210pmc: PMC7070845pubmed: 32012670google scholar: lookup
  23. Hintze S, Smith S, Patt A, Bachmann I, Würbel H. Are Eyes a Mirror of the Soul? What Eye Wrinkles Reveal about a Horse's Emotional State.. PLoS One 2016;11(10):e0164017.
    doi: 10.1371/journal.pone.0164017pmc: PMC5061373pubmed: 27732647google scholar: lookup
  24. Doherty O, Casey V, McGreevy P, Arkins S. Noseband Use in Equestrian Sports - An International Study.. PLoS One 2017;12(1):e0169060.
    doi: 10.1371/journal.pone.0169060pmc: PMC5207535pubmed: 28045961google scholar: lookup
  25. Fenner K, Yoon S, White P, Starling M, McGreevy P. The Effect of Noseband Tightening on Horses' Behavior, Eye Temperature, and Cardiac Responses.. PLoS One 2016;11(5):e0154179.
    doi: 10.1371/journal.pone.0154179pmc: PMC4854461pubmed: 27140187google scholar: lookup
  26. Anderson P.H, Gleerup K.B, Wathan J, Coles B, Kjellström H, Broomé S, Lee Y.J, Rashid M, Sonder C, Rosenberger E. Can a machine learn to see horse pain? An interdisciplinary approach towards automated decoding of facial expressions of pain in the horse. Proceedings of the Measuring Behavioral Research; Manchester, UK. 5–8 June 2018; p. 209526193.
  27. Bussières G, Jacques C, Lainay O, Beauchamp G, Leblond A, Cadoré JL, Desmaizières LM, Cuvelliez SG, Troncy E. Development of a composite orthopaedic pain scale in horses.. Res Vet Sci 2008 Oct;85(2):294-306.
    doi: 10.1016/j.rvsc.2007.10.011pubmed: 18061637google scholar: lookup
  28. Lindegaard C, Gleerup KB, Thomsen MH, Martinussen T, Jacobsen S, Andersen PH. Anti-inflammatory effects of intra-articular administration of morphine in horses with experimentally induced synovitis.. Am J Vet Res 2010 Jan;71(1):69-75.
    doi: 10.2460/ajvr.71.1.69pubmed: 20043783google scholar: lookup
  29. Kil N, Ertelt K, Auer U. Development and Validation of an Automated Video Tracking Model for Stabled Horses.. Animals (Basel) 2020 Nov 30;10(12).
    doi: 10.3390/ani10122258pmc: PMC7760072pubmed: 33266297google scholar: lookup
  30. Kim H.I, Moon J.Y, Park J.Y. Research trends for deep learning-based high-performance face recognition technology. ETRI J. 2018;33:43–53.
    doi: 10.22648/ETRI.2018.J.330405google scholar: lookup
  31. Graving JM, Chae D, Naik H, Li L, Koger B, Costelloe BR, Couzin ID. DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning.. Elife 2019 Oct 1;8.
    doi: 10.7554/eLife.47994pmc: PMC6897514pubmed: 31570119google scholar: lookup
  32. Mathis A, Mamidanna P, Cury KM, Abe T, Murthy VN, Mathis MW, Bethge M. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning.. Nat Neurosci 2018 Sep;21(9):1281-1289.
    doi: 10.1038/s41593-018-0209-ypubmed: 30127430google scholar: lookup
  33. Mathis MW, Mathis A. Deep learning tools for the measurement of animal behavior in neuroscience.. Curr Opin Neurobiol 2020 Feb;60:1-11.
    doi: 10.1016/j.conb.2019.10.008pubmed: 31791006google scholar: lookup
  34. Burla J.B, Ostertag A, Schulze Westerath H.S, Hillmann E. Gait determination and activity measurement in horses using an accelerometer. Comput. Electron. Agric. 2014;102:127–133.
    doi: 10.1016/j.compag.2014.01.001google scholar: lookup
  35. Eerdekens A, Deruyck M, Fontaine J, Martens L, Poorter E.D, Joseph W. Automatic equine activity detection by convolutional neural networks using accelerometer data. Comput. Electron. Agric. 2020;168:105–139.
    doi: 10.1016/j.compag.2019.105139google scholar: lookup
  36. Wathan J, Burrows AM, Waller BM, McComb K. EquiFACS: The Equine Facial Action Coding System.. PLoS One 2015;10(8):e0131738.
    doi: 10.1371/journal.pone.0131738pmc: PMC4526551pubmed: 26244573google scholar: lookup
  37. Dalla Costa E, Minero M, Lebelt D, Stucke D, Canali E, Leach MC. Development of the Horse Grimace Scale (HGS) as a pain assessment tool in horses undergoing routine castration.. PLoS One 2014;9(3):e92281.
    doi: 10.1371/journal.pone.0092281pmc: PMC3960217pubmed: 24647606google scholar: lookup
  38. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Proceedings of the Computer Vision and Pattern Recognition; Columbus, OH, USA. 10 April 2015.
    doi: 10.48550/arXiv.1409.1556google scholar: lookup
  39. Minaee S, Minaei M, Abdolrashidi A. Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network.. Sensors (Basel) 2021 Apr 27;21(9).
    doi: 10.3390/s21093046pmc: PMC8123912pubmed: 33925371google scholar: lookup
  40. Zhu X, Ye S, Zhao L, Dai Z. Hybrid Attention Cascade Network for Facial Expression Recognition.. Sensors (Basel) 2021 Mar 12;21(6).
    doi: 10.3390/s21062003pmc: PMC8002145pubmed: 33809038google scholar: lookup
  41. Mahmood A, Ospina AG, Bennamoun M, An S, Sohel F, Boussaid F, Hovey R, Fisher RB, Kendrick GA. Automatic Hierarchical Classification of Kelps Using Deep Residual Features.. Sensors (Basel) 2020 Jan 13;20(2).
    doi: 10.3390/s20020447pmc: PMC7013955pubmed: 31941132google scholar: lookup
  42. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Las Vegas, NV, USA. 27–30 June 2016; pp. 770–778.
    doi: 10.1109/CVPR.2016.90google scholar: lookup
  43. Tan M, Pang R, Le Q.V. EfficientDet: Scalable and Efficient object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Seattle, WA, USA. 13–19 June 2020.
    doi: 10.1109/CVPR42600.2020.01079google scholar: lookup
  44. Lin M, Chen Q, Yan S. Network in network. Proceedings of the International Conference on Learning Representations 2014; Banff, AB, Canada. 28 February 2014.
    doi: 10.48550/arXiv.1911.09070google scholar: lookup
  45. Szegedy C.W, Liu Y, Jia P, Sermanet S, Reed D, Anguelov D, Erhan V, Vanhoucke V, Rabinovich A. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Boston, MA, USA. 7–12 June 2015.
    doi: 10.1109/CVPR.2015.7298594google scholar: lookup
  46. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Boston, MA, USA. 7–12 June 2015.
    doi: 10.1109/CVPR.2016.319google scholar: lookup
  47. Zeiler M.D, Fergus R. Visualizing and understanding convolutional networks. Proceedings of the 13th European Conference; Zurich, Switzerland. 6–12 September 2014; pp. 818–833. Lecture Notes in Computer Science.
    doi: 10.48550/arXiv.1311.2901google scholar: lookup
  48. Egenvall A, Penell JC, Bonnett BN, Olson P, Pringle J. Mortality of Swedish horses with complete life insurance between 1997 and 2000: variations with sex, age, breed and diagnosis.. Vet Rec 2006 Mar 25;158(12):397-406.
    doi: 10.1136/vr.158.12.397pubmed: 16565338google scholar: lookup
  49. Stover S.M. The epidemiology of Thoroughbred racehorse injuries. Clin. Tech. Equine Pract. 2003;2:312–322.
    doi: 10.1053/j.ctep.2004.04.003google scholar: lookup
  50. Logan AA, Nielsen BD. Training Young Horses: The Science behind the Benefits.. Animals (Basel) 2021 Feb 9;11(2).
    doi: 10.3390/ani11020463pmc: PMC7916178pubmed: 33572461google scholar: lookup
  51. Lucey P, Cohn J.F, Kanade T, Saragih J, Ambadar Z, Matthews I. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition—Workshops; San Francisco, CA, USA. 13–18 June 2010; pp. 94–101.
    doi: 10.1109/CVPRW.2010.5543262google scholar: lookup
  52. Lucey P, Cohn J.F, Prkachin K.M, Solomon P.E, Matthews I. 2011 Painful data: The UNBC-McMaster shoulder pain expression archive database. Proceedings of the 2011 International Conference on Automatic Face & Gesture Recognition; Santa Barbara, CA, USA. 21–25 March 2011; pp. 57–64.
    doi: 10.1109/FG.2011.5771462google scholar: lookup
  53. Zhang X, Yin L, Cohn J.F, Canavan S, Reale M, Horowitz A, Liu P, Girard J.M. BP4D-spontaneous: A high-resolution spontaneous 3D dynamic facial expression database. Image. Vis. Comput. 2014;32:692–706.
    doi: 10.1016/j.imavis.2014.06.002google scholar: lookup
  54. LeCun Y. Generalization and network design strategies. Conn. Perspect. 1989;119:143–155.
  55. Pooya K, Paine T, Huang T. Do deep neural networks learn facial action units when doing expression recognition?. Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW); Santiago, Chile. 7–13 December 2015.
    doi: 10.1109/ICCVW.2015.12google scholar: lookup
  56. Tzirakis P, Trigeorgis G, Nicolaou M.A, Schuller B.W, Zafeiriou S. End-to-end multimodal emotion recognition using deep neural networks. IEEE J. Sel. Top. Signal. Process. 2017;11:1301–1309.
    doi: 10.1109/JSTSP.2017.2764438google scholar: lookup
  57. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition; Columbus, OH, USA. 23–28 June 2014.
    doi: 10.1109/CVPR.2014.81google scholar: lookup
  58. Krizhevsky A, Sutskever I, Hinton G.E. Imagenet classification with deep convolutional neural networks. Commun. ACM. 2017;60:84–90.
    doi: 10.1145/3065386google scholar: lookup
  59. Zhou B, Lapedriza A, Xiao J, Torralba A, Oliva A. Learning deep features for scene recognition using places database. Proceedings of the 27th International Conference on Neural Information processing Systems; Montreal, QC, Canada. 8–13 December 2014; pp. 487–495.
  60. Bergamo A, Bazzani L, Anguelov D, Torresani L. Self-taught object localization with deep networks. Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision; Lake Placid, NY, USA. 7–10 March 2016.
    doi: 10.1109/WACV.2016.7477688google scholar: lookup
  61. Cinbis RG, Verbeek J, Schmid C. Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning.. IEEE Trans Pattern Anal Mach Intell 2017 Jan;39(1):189-203.
    doi: 10.1109/TPAMI.2016.2535231pubmed: 26930676google scholar: lookup
  62. Oquab M, Bottou L, Laptev I, Sivic J. Learning and transferring mid-level image representations using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Columbus, OH, USA. 23–28 June 2014.
    doi: 10.1109/CVPR.2014.222google scholar: lookup
  63. Oquab M, Bottou L, Laptev I, Sivic J. Is object localization for free? weakly supervised learning with convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Boston, MA, USA. 7–12 June 2015.
    doi: 10.1109/CVPR.2015.7298668google scholar: lookup
  64. Cohn F.J, Zlochower A. A computerized analysis of facial expression: Feasibility of automated discrimination. Am. Psychol. Soc. 1995;2:6.
  65. Dyson S, Berger J.M, Ellis A.D, Mullard J. Can the presence of musculoskeletal pain be determined from the facial expressions of ridden horses (FEReq)?. J. Vet. Behav. 2017;19:78–89.
    doi: 10.1016/j.jveb.2017.03.005google scholar: lookup
  66. Dalla Costa E, Stucke D, Dai F, Minero M, Leach MC, Lebelt D. Using the Horse Grimace Scale (HGS) to Assess Pain Associated with Acute Laminitis in Horses (Equus caballus).. Animals (Basel) 2016 Aug 3;6(8).
    doi: 10.3390/ani6080047pmc: PMC4997272pubmed: 27527224google scholar: lookup
  67. Shin S.K, Kim S.M, Lioyd S, Cho G.J. Prevalence of hoof disorders in horses in South Korea. Open Agric. J. 2020;14:25–29.

Citations

This article has been cited 0 times.
Subscribe to our newsletter
Join 99,344 horse owners like you who receive our email updates on horse health and nutrition.