Analyze Diet
PloS one2022; 17(3); e0263854; doi: 10.1371/journal.pone.0263854

Sharing pain: Using pain domain transfer for video recognition of low grade orthopedic pain in horses.

Abstract: Orthopedic disorders are common among horses, often leading to euthanasia, which often could have been avoided with earlier detection. These conditions often create varying degrees of subtle long-term pain. It is challenging to train a visual pain recognition method with video data depicting such pain, since the resulting pain behavior also is subtle, sparsely appearing, and varying, making it challenging for even an expert human labeller to provide accurate ground-truth for the data. We show that a model trained solely on a dataset of horses with acute experimental pain (where labeling is less ambiguous) can aid recognition of the more subtle displays of orthopedic pain. Moreover, we present a human expert baseline for the problem, as well as an extensive empirical study of various domain transfer methods and of what is detected by the pain recognition method trained on clean experimental pain in the orthopedic dataset. Finally, this is accompanied with a discussion around the challenges posed by real-world animal behavior datasets and how best practices can be established for similar fine-grained action recognition tasks. Our code is available at https://github.com/sofiabroome/painface-recognition.
Publication Date: 2022-03-04 PubMed ID: 35245288PubMed Central: PMC8896717DOI: 10.1371/journal.pone.0263854Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
  • Journal Article
  • Research Support
  • Non-U.S. Gov't

Summary

This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.

This study explores the use of a computer model trained to recognize acute pain in horses from video data as a tool for detecting subtler, chronic pain associated with orthopedic conditions. Via domain transfer methods and baseline expertise, the researchers demonstrate the potential of this technology as an early detection tool for equine orthopedic disorders.

Clarification of Research Purpose

  • This research aims to create a machine learning model that can identify subtle signs of long-term orthopedic pain in horses.
  • The need for this arises from the difficulty of identifying such pain early on, often leading to late-stage diagnosis and substantial consequences such as euthanasia.
  • Currently, consistently accurate labels for training such a model are difficult to establish since it’s challenging even for human experts to distinguish these subtle pain indicators.

Data and Model Training

  • A dataset of video footage exhibiting horses with acute, experimental pain was used to train the model. This was done since acute pain manifestations are less ambiguous and are easier to label than subtle, chronic pain.
  • The hope was that a model trained in this manner could then identify more nuanced and varied signs of long-term orthopedic pain in horses by transferring knowledge from the acute pain domain.

Application & Study Findings

  • The study presents an empirical investigation into various domain transfer methods, that is, how to best train the model on acute pain data to recognize chronic orthopedic pain.
  • The effectiveness of the recognition model, trained on clear experimental pain, is then tested on the orthopedic dataset.
  • The results revealed the potential of the model in identifying orthopedic pain, implicitly demonstrating the practical use of this technology for early diagnosis.

Significance and Future Remarks

  • The findings open up a discussion on the challenges inherent to using machine learning on real-world animal behavior datasets and how best to address these.
  • With this study, best practices can be established for similar fine-grained action recognition tasks in the future.
  • Overall, this research has important implications for veterinary medicine and for furthering humane treatment of animals, by aiding in the early detection and treatment of painful medical conditions.
  • The code used in this research is publicly available, fostering further innovation and application by other researchers in this field.

Cite This Article

APA
Broomé S, Ask K, Rashid-Engström M, Haubro Andersen P, Kjellström H. (2022). Sharing pain: Using pain domain transfer for video recognition of low grade orthopedic pain in horses. PLoS One, 17(3), e0263854. https://doi.org/10.1371/journal.pone.0263854

Publication

ISSN: 1932-6203
NlmUniqueID: 101285081
Country: United States
Language: English
Volume: 17
Issue: 3
Pages: e0263854

Researcher Affiliations

Broomé, Sofia
  • Division of Robotics, Perception and Learning, KTH Royal Institute of Technology, Stockholm, Sweden.
Ask, Katrina
  • Department of Anatomy, Physiology and Biochemistry, Swedish University of Agricultural Sciences, Uppsala, Sweden.
Rashid-Engström, Maheen
  • Department of Computer Science, University of California, Davis, California, United States of America.
  • Univrses, Stockholm, Sweden.
Haubro Andersen, Pia
  • Department of Clinical Sciences, Swedish University of Agricultural Sciences, Uppsala, Sweden.
Kjellström, Hedvig
  • Division of Robotics, Perception and Learning, KTH Royal Institute of Technology, Stockholm, Sweden.
  • Silo AI, Stockholm, Sweden.

MeSH Terms

  • Animals
  • Communications Media
  • Horses
  • Pain / veterinary

Conflict of Interest Statement

The authors have declared that no competing interests exist.

References

This article includes 76 references
  1. Taylor PM, Pascoe PJ, Mama KR. Diagnosing and treating pain in the horse. Where are we today?. Veterinary Clinics of North America—Equine Practice 2002;18(1):1–19.
    doi: 10.1016/S0749-0739(02)00009-3pubmed: 12064173google scholar: lookup
  2. Torcivia C, McDonnell S. In-Person Caretaker Visits Disrupt Ongoing Discomfort Behavior in Hospitalized Equine Orthopedic Surgical Patients. Animals 2020;10(2).
    doi: 10.3390/ani10020210pmc: PMC7070845pubmed: 32012670google scholar: lookup
  3. Penell JC, Egenvall A, Bonnett BN, Olson P, Pringle J. Specific causes of morbidity among Swedish horses insured for veterinary care between 1997 and 2000. Veterinary Record 2005;157:470—477.
    doi: 10.1136/vr.157.16.470pubmed: 16227382google scholar: lookup
  4. Pollard D, Wylie CE, Newton JR, Verheyen KLP. Factors Associated with Euthanasia in Horses and Ponies Enrolled in a Laminitis Cohort Study in Great Britain. Preventive Veterinary Medicine 2020;174(November 2019):104833.
  5. Slayter J, Taylor G. National Equine Health Survey (NEHS) 2018. 2018.
    doi: 10.1136/vr.f4967google scholar: lookup
  6. Sneddon LU, Elwood RW, Adamo SA, Leach MC. Defining and assessing animal pain. Animal Behaviour 97:201–212.
  7. Ashley FH, Waterman-Pearson AE, Whay HR. Behavioural assessment of pain in horses and donkeys: application to clinical practice and future studies. Equine Veterinary Journal 2005;37(6):565–575.
    doi: 10.2746/042516405775314826pubmed: 16295937google scholar: lookup
  8. Loeser JD, Treede RD. The Kyoto protocol of IASP Basic Pain Terminology. Pain 2008;137(3):473–477.
    doi: 10.1016/j.pain.2008.04.025pubmed: 18583048google scholar: lookup
  9. Romero R, Souzdalnitski D, Banack T. Ischemic and Visceral Pain. Springer Verlag; New York; 2011.
  10. Gu C, Sun C, Ross DA, Vondrick C, Pantofaru C, Li Y. AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2018.
  11. Kay W, Carreira J, Simonyan K, Zhang B, Hillier C, Vijayanarasimhan S. The Kinetics Human Action Video Dataset. CoRR 2017;abs/1705.06950.
  12. Soomro K, Zamir AR, Shah M. UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. CoRR 2012;abs/1212.0402.
  13. Mahdisoltani F, Berger G, Gharbieh W, Fleet DJ, Memisevic R. Fine-grained Video Classification and Captioning. CoRR 2018;abs/1804.09235.
  14. Li Y, Li Y, Vasconcelos N. RESOUND: Towards Action Recognition without Representation Bias. In: Proceedings of the European Conference on Computer Vision (ECCV); 2018.
  15. Shao D, Zhao Y, Dai B, Lin D. FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2020.
  16. Ask K, Rhodin M, Tamminen LM, Hernlund E, Haubro Andersen P. Identification of Body Behaviors and Facial Expressions Associated with Induced Orthopedic Pain in Four Equine Pain Scales. Animals 2020;10(11).
    doi: 10.3390/ani10112155pmc: PMC7699379pubmed: 33228117google scholar: lookup
  17. Costa ED, Minero M, Lebelt D, Stucke D, Canali E, Leach MC. Development of the Horse Grimace Scale (HGS) as a Pain Assessment Tool in Horses Undergoing Routine Castration. PLoS ONE 2014;9(3).
  18. Dalla Costa E, Stucke D, Dai F, Minero M, Leach MC, Lebelt D. Using the Horse Grimace Scale (HGS) to Assess Pain Associated with Acute Laminitis in Horses (Equus Caballus). Animals 2016;6(47):1–9.
    doi: 10.3390/ani6080047pmc: PMC4997272pubmed: 27527224google scholar: lookup
  19. Gleerup KB, Lindegaard C. Recognition and quantification of pain in horses: A tutorial review. Equine Veterinary Education 2016;28(1):47–57.
    doi: 10.1111/eve.12383google scholar: lookup
  20. van Loon JPAM, Van Dierendonck MC. Objective pain assessment in horses (2014–2018). Veterinary Journal 2018;242:1–7.
    doi: 10.1016/j.tvjl.2018.10.001pubmed: 30503538google scholar: lookup
  21. Gleerup KB, Forkman B, Lindegaard C, Andersen PH. An equine pain face. Veterinary Anaesthesia and Analgesia 2015;42.
    doi: 10.1111/vaa.12212pmc: PMC4312484pubmed: 25082060google scholar: lookup
  22. Broomé S, Gleerup KB, Andersen PH, Kjellström H. Dynamics Are Important for the Recognition of Equine Pain in Video. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2019.
  23. Islam A, Radke R. Weakly Supervised Temporal Action Localization Using Deep Metric Learning. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV); 2020.
  24. Rashid M, Kjellström H, Lee YJ. Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks. In: The IEEE Winter Conference on Applications of Computer Vision; 2020. p. 615–624.
  25. Paul S, Roy S, Roy-Chowdhury AK. W-TALC: Weakly-supervised Temporal Activity Localization and Classification. In: Proceedings of the European Conference on Computer Vision (ECCV); 2018. p. 563–579.
  26. Nguyen P, Liu T, Prasad G, Han B. Weakly Supervised Action Localization by Sparse Temporal Pooling Network. In: CVPR; 2018.
  27. Wang L, Xiong Y, Lin D, Van Gool L. Untrimmednets for weakly supervised action recognition and detection. In: CVPR; 2017.
  28. Ilse M, Tomczak J, Welling M. Attention-based Deep Multiple Instance Learning. In: Dy J, Krause A, editors. 35th International Conference on Machine Learning, ICML 2018. 35th International Conference on Machine Learning, ICML 2018. International Machine Learning Society (IMLS); 2018. p. 3376–3391.
  29. Li X, Lang Y, Chen Y, Mao X, He Y, Wang S. Sharp Multiple Instance Learning for DeepFake Video Detection. Proceedings of the 28th ACM International Conference on Multimedia 2020.
    doi: 10.1145/3394171.3414034google scholar: lookup
  30. Wang X, Yan Y, Tang P, Bai X, Liu W. Revisiting Multiple Instance Neural Networks. arXiv preprint arXiv:161002501 2016.
  31. Cinbis RG, Verbeek J, Schmid C. Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 2017;39(1):189–203.
    doi: 10.1109/TPAMI.2016.2535231pubmed: 26930676google scholar: lookup
  32. Carreira J, Zisserman A. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. In: CVPR; 2017.
  33. Zhai Y, Wang L, Liu Z, Zhang Q, Hua G, Zheng N. Action Coherence Network for Weakly Supervised Temporal Action Localization. In: ICIP; 2019.
  34. Singh KK, Lee YJ. Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization. In: ICCV; 2017.
  35. Zeng R, Gan C, Chen P, Huang W, Wu Q, Tan M. Breaking winner-takes-all: Iterative-winners-out networks for weakly supervised temporal action localization. IEEE Transactions on Image Processing 2019.
    pubmed: 31217119
  36. Arnab A, Sun C, Nagrani A, Schmid C. Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos. In: Vedaldi A, Bischof H, Brox T, Frahm JM, editors. Computer Vision—ECCV 2020. Cham: Springer International Publishing; 2020. p. 751–768.
  37. Yang P, Hu VT, Mettes P, Snoek CGM. Localizing the Common Action Among a Few Videos. In: Vedaldi A, Bischof H, Brox T, Frahm JM, editors. Computer Vision—ECCV 2020. Cham: Springer International Publishing; 2020. p. 505–521.
  38. Lu Y, Mahmoud M, Robinson P. Estimating Sheep Pain Level Using Facial Action Unit Detection. In: 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017); 2017.
  39. Hummel HI, Pessanha F, Salah A, van Loon TM, Veltkamp RC. Automatic Pain Detection on Horse and Donkey Faces. In: 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) (FG). Los Alamitos, CA, USA: IEEE Computer Society; 2020. p. 717–724.
  40. Pessanha P, McLennan K, Mahmoud M. Towards automatic monitoring of disease progression in sheep: A hierarchical model for sheep facial expressions analysis from video. In: 15th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2020); 2020.
  41. Tuttle AH, Molinaro MJ, Jethwa JF, Sotocinal SG, Prieto JC, Styner MA. A deep neural network to assess spontaneous pain from mouse facial expressions. Molecular Pain 2018;14:1744806918763658.
    doi: 10.1177/1744806918763658pmc: PMC5858615pubmed: 29546805google scholar: lookup
  42. Andresen N, Wöllhaf M, Hohlbaum K, Lewejohann L, Hellwich O, Thöne-Reineke C. Towards a fully automated surveillance of well-being status in laboratory mice using deep learning: Starting with facial expression analysis. PLOS ONE 2020;15(4):1–23.
  43. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) 2015;115(3):211–252.
    doi: 10.1007/s11263-015-0816-ygoogle scholar: lookup
  44. He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. p. 770–778.
  45. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the Inception Architecture for Computer Vision. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition; 2016.
  46. Lencioni GC, de Sousa RV, de Souza Sardinha EJ, Corrêa RR, Zanella AJ. Pain assessment in horses using automatic facial expression recognition through deep learning-based modeling. PLOS ONE 2021;16(10):1–12.
  47. Li Z, Broomé S, Andersen PH, Kjellström H. Automated Detection of Equine Facial Action Units. ArXiv 2021;abs/2102.08983.
  48. Wathan J, Burrows AM, Waller BM, McComb K. EquiFACS: The Equine Facial Action Coding System. PLOS ONE 2015;10(8):1–35.
  49. Rashid M, Broomé S, Ask K, Hernlund E, Andersen PH, Kjellström H. Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation. In: Winter Conference on Applications of Computer Vision (WACV), to appear.; 2022.
  50. Cohen S, Beths T. Grimace Scores: Tools to Support the Identification of Pain in Mammals Used in Research. Animals 2020;10(10).
    doi: 10.3390/ani10101726pmc: PMC7598254pubmed: 32977561google scholar: lookup
  51. Mogil JS, Pang DSJ, Silva Dutra GG, Chambers CT. The Development and Use of Facial Grimace Scales for Pain Measurement in Animals. Neuroscience & Biobehavioral Reviews 2020;116:480–493.
  52. van Weeren PR, de Grauw JC. Pain in Osteoarthritis. Veterinary Clinics of North America: Equine Practice 2010;26(3):619–642.
    pubmed: 21056303
  53. Schaible H. Mechanisms of Chronic Pain in Osteoarthritis. Current Rheumatology Reports 2012;14:549–556.
    doi: 10.1007/s11926-012-0279-xpubmed: 22798062google scholar: lookup
  54. Van de Water E, Oosterlinck M, Korthagen NM, Duchateau L, Dumoulin M, van Weeren PR. The lipopolysaccharide model for the experimental induction of transient lameness and synovitis in Standardbred horses. The Veterinary Journal 2021;270:105626.
    doi: 10.1016/j.tvjl.2021.105626pubmed: 33641810google scholar: lookup
  55. Bussiéres G, Jacques C, Lainay O, Beauchamp G, Leblond A, Cadoré JL. Development of a composite orthopaedic pain scale in horses. Res Vet Sci 2008;85(2).
    pubmed: 18061637
  56. Shi X, Chen Z, Wang H, Yeung D, Wong W, Woo W. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R, editors. Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Q, Canada; 2015. p. 802–810.
  57. Hochreiter S, Schmidhuber J. Long Short-Term Memory. Neural Comput 1997;9(8):1735–1780.
    doi: 10.1162/neco.1997.9.8.1735pubmed: 9377276google scholar: lookup
  58. Wiese AJ, Gaynor JS, Muir W. Assessing Pain: Pain Behaviors, in Handbook of veterinary pain management (Third Edition). 2015.
  59. de Camp Nora V, LW Mechthild, G CIE, Thöne-Reineke Christa, B J. EEG based assessment of stress in horses: A pilot study. PeerJ 2020;8:e8629:1–15.
    doi: 10.7717/peerj.8629pmc: PMC7227666pubmed: 32435527google scholar: lookup
  60. Andreassen SM, Vinther AML, Nielsen SS, Andersen PH, Tnibar A, Kristensen AT. Changes in concentrations of haemostatic and inflammatory biomarkers in synovial fluid after intra-articular injection of lipopolysaccharide in horses. BMC Veterinary Research 2017;13(1):182.
    doi: 10.1186/s12917-017-1089-1pmc: PMC5477303pubmed: 28629364google scholar: lookup
  61. Rashid M, Silventoinen A, Gleerup KB, Andersen PH. Equine Facial Action Coding System for determination of pain-related facial responses in videos of horses. PLOS ONE 2020;15(11):1–18.
  62. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision 2017;2017-Octob:618–626.
    doi: 10.1109/ICCV.2017.74google scholar: lookup
  63. Andersen PH, Broomé S, Rashid M, Lundblad J, Ask K, Li Z. Towards Machine Recognition of Facial Expressions of Pain in Horses. Animals 2021;11(6), 1643.
    doi: 10.3390/ani11061643pmc: PMC8229776pubmed: 34206077google scholar: lookup
  64. Waran N, Williams V, Clarke N, Bridge I. Recognition of pain and use of analgesia in horses by veterinarians in New Zealand. New Zealand Veterinary Journal 2010;58:274—280.
    doi: 10.1080/00480169.2010.69402pubmed: 21151212google scholar: lookup
  65. Farina S, Valeriani M, Rosso T, Aglioti SM, Tinazzi M. Transient inhibition of the human motor cortex by capsaicin-induced pain. A study with transcranial magnetic stimulation. Neuroscience Letters 2001;314:97–101.
    doi: 10.1016/S0304-3940(01)02297-2pubmed: 11698155google scholar: lookup
  66. Tuveson B, Leffler AS, Hansson PT. Time dependant differences in pain sensitivity during unilateral ischemic pain provocation in healthy volunteers. European Journal of Pain 2006;10.
    doi: 10.1016/j.ejpain.2005.03.010pubmed: 15919219google scholar: lookup
  67. Lutgendorf SK, Logan H, Kirchner HL, Rothrock NE, Svengalis S, Iverson K. Effects of Relaxation and Stress on the Capsaicin-Induced Local Inflammatory Response. Psychosomatic Medicine 2000;62:524–534.
  68. Rhodin M, Persson-Sjodin E, Egenvall A, Bragança FMS, Pfau T, Roepstorff L. Vertical movement symmetry of the withers in horses with induced forelimb and hindlimb lameness at trot. Equine Veterinary Journal 2018;50:818—824.
    doi: 10.1111/evj.12844pmc: PMC6175082pubmed: 29658147google scholar: lookup
  69. McLennan KM, Rebelo CJB, Corke MJ, Holmes MA, Leach MC, Constantino-Casas F. Development of a facial expression scale using footrot and mastitis as models of pain in sheep. Applied Animal Behaviour Science 2016;176:19–26.
  70. Hayashi K, Ikemoto T, Ueno T, Arai YCP, Shimo K, Nishihara M. Discordant Relationship Between Evaluation of Facial Expression and Subjective Pain Rating Due to the Low Pain Magnitude. Basic and Clinical Neuroscience Journal 2018;9(1).
    doi: 10.29252/nirp.bcn.9.1.43pmc: PMC6015640pubmed: 29942439google scholar: lookup
  71. Dalla Costa E, Pascuzzo R, Leach MC, Dai F, Lebelt D, Vantini S. Can grimace scales estimate the pain status in horses and mice? A statistical approach to identify a classifier. PLOS ONE 2018;13(8):1–17.
  72. Pilitsis JG, Fahey M, Custozzo A, Chakravarthy K, Capobianco R. Composite Score Is a Better Reflection of Patient Response to Chronic Pain Therapy Compared With Pain Intensity Alone. Neuromodulation: Technology at the Neural Interface 2021;24(1):68–75.
    doi: 10.1111/ner.13212pubmed: 32592618google scholar: lookup
  73. Trindade PHE, Taffarel MO, Luna SPL. Spontaneous Behaviors of Post-Orchiectomy Pain in Horses Regardless of the Effects of Time of Day, Anesthesia, and Analgesia. Animals 2021;11(6).
    doi: 10.3390/ani11061629pmc: PMC8230028pubmed: 34072875google scholar: lookup
  74. Cao J, Tang H, Fang HS, Shen X, Lu C, Tai YW. Cross-Domain Adaptation for Animal Pose Estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); 2019.
  75. Mathis A, Biasi T, Schneider S, Yuksekgonul M, Rogers B, Bethge M. Pretraining Boosts Out-of-Domain Robustness for Pose Estimation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV); 2021. p. 1859–1868.
  76. Lundblad J, Rashid M, Rhodin M, Andersen PH. Effect of transportation and social isolation on facial expressions of healthy horses. PLoS One 2021;16(6).