Analyze Diet
Animals : an open access journal from MDPI2021; 11(8); 2303; doi: 10.3390/ani11082303

Development and Validation of a Test for the Classification of Horses as Broken or Unbroken.

Abstract: Regulation EC 1/2005 has stricter rules for transportation of unbroken (untamed) vs. broken (tamed) horses, but does not provide adequate tools for their identification. This study aimed to develop and validate such a tool. A behavioural test (Broken/Unbroken Test (BUT)) based on approaching, haltering, and leading was applied to 100 horses. Physiological and additional behavioural data were also collected, and the horses' status (broken/unbroken) was assessed by the expert who administered the BUT. Each horse's behaviour during the BUT was scored by four trained observers blinded to the horse's history. The BUT score showed excellent inter-observer, intra-observer, and test-retest reliability (all intraclass correlation coefficients (ICCs) > 0.75). It was also negatively associated with respiratory rate, avoidance distance, and time needed to approach, halter, and lead the horse ( < 0.05 for all). The optimal BUT score cut-off for discrimination between broken and unbroken horses (gold standard: expert judgment) showed 97.8% sensitivity and 97.3% specificity. There was almost perfect agreement between BUT-based and expert classification of horses (ICC = 0.940). These findings confirm the BUT's construct and criterion validity. The BUT could provide officials with a feasible, reliable, and valid tool to identify a horse's broken/unbroken status and, consequently, direct stakeholders towards correct transport procedures.
Publication Date: 2021-08-04 PubMed ID: 34438758PubMed Central: PMC8388372DOI: 10.3390/ani11082303Google Scholar: Lookup
The Equine Research Bank provides access to a large database of publicly available scientific literature. Inclusion in the Research Bank does not imply endorsement of study methods or findings by Mad Barn.
  • Journal Article

Summary

This research summary has been generated with artificial intelligence and may contain errors and omissions. Refer to the original study to confirm details provided. Submit correction.

The research aimed to develop and validate a reliable tool for differentiating between tamed (broken) and untamed (unbroken) horses, adhering to the specific transportation regulations for each.

Study Design and Methodology

  • The researchers designed a behavioral test known as the Broken/Unbroken Test (BUT) that involved approaching, haltering, and leading a horse.
  • The BUT was performed on 100 horses, and additional physiological and behavioral data were collected.
  • The horses’ status, broken or unbroken, was evaluated by an expert who administered the BUT.
  • The behavior of every horse during the BUT was then scored by four trained observers who were unaware of the horse’s individual history.

Validity and Reliability of the BUT

  • The scores granted during the BUT demonstrated excellent inter-observer, intra-observer, and test-retest reliability, with all intraclass correlation coefficients (ICCs) exceeding 0.75.
  • There was a negative correlation found between the BUT score and a horse’s respiratory rate, avoidance distance, and the time it took to approach, halter, and lead the horse.
  • The optimal CUT score cutoff proved efficient in distinguishing between broken and unbroken horses, demonstrating a 97.8% sensitivity and 97.3% specificity in the gold standard: expert judgment.

Conclusions and Implications

  • The results displayed almost perfect concurrence between the BUT-based classification of horses and expert classification, supporting the BUT’s construct and criterion validity.
  • The BUT could potentially serve as a reliable and valid tool for officials to determine a horse’s status —broken or unbroken.
  • This could guide stakeholders towards the correct transport procedures, in line with Regulation EC 1/2005, which calls for stricter rules when transporting unbroken horses.

Cite This Article

APA
Menchetti L, Dalla Costa E, Minero M, Padalino B. (2021). Development and Validation of a Test for the Classification of Horses as Broken or Unbroken. Animals (Basel), 11(8), 2303. https://doi.org/10.3390/ani11082303

Publication

ISSN: 2076-2615
NlmUniqueID: 101635614
Country: Switzerland
Language: English
Volume: 11
Issue: 8
PII: 2303

Researcher Affiliations

Menchetti, Laura
  • Department of Agricultural and Food Sciences, Alma Mater Studiorum-University of Bologna, 40127 Bologna, Italy.
Dalla Costa, Emanuela
  • Department of Veterinary Medicine, University of Milan, 26900 Lodi, Italy.
Minero, Michela
  • Department of Veterinary Medicine, University of Milan, 26900 Lodi, Italy.
Padalino, Barbara
  • Department of Agricultural and Food Sciences, Alma Mater Studiorum-University of Bologna, 40127 Bologna, Italy.

Grant Funding

  • d21eq-522 / ANIMALS' ANGELS, WORLD HORSE WELFARE, ANIMAL MORRIS FOUNDATION

Conflict of Interest Statement

The authors declare no conflict of interest.

References

This article includes 62 references
  1. Waran N, Leadon D, Friend T. The Effects of Transportation on the Welfare of Horses. 2007; pp. 125–150.
  2. European Parliament. Report on Responsible Ownership and Care of Equidae (2016/2078(INI)). 2016.
  3. . Scientific Opinion Concerning the Welfare of Animals during Transport. EFSA J 2011;9:1966.
    doi: 10.2903/j.efsa.2011.1966google scholar: lookup
  4. Baltussen W, Gebrensbet G, De Roest K. Study on the Impact of Regulation (EC) No. 1/2005 on the Protection of Animals during Transport. 2011 :119.
  5. Knowles TG, Brown SN, Pope SJ, Nicol CJ, Warriss PD, Weeks CA. The response of untamed (unbroken) ponies to conditions of road transport. Anim Welf 2010;19:1–15.
  6. Sulzberger AG. Slaughter of Horses Goes On, Just Not in U.S.. The New York Times 2011 Oct 23.
  7. Miranda-de la Lama GC, Gonzales-Castro CA, Gutierrez-Piña FJ, Villarroel M, Maria GA, Estévez-Moreno LX. Welfare of horses from Mexico and the United States of America transported for slaughter in Mexico: Fitness profiles for transport and pre-slaughter logistics.. Prev Vet Med 2020 Jul;180:105033.
  8. Roy RC, Cockram MS. Patterns and durations of journeys by horses transported from the USA to Canada for slaughter.. Can Vet J 2015 Jun;56(6):581-6.
    pmc: PMC4431154pubmed: 26028678
  9. Padalino B, Raidal SL. Effects of Transport Conditions on Behavioural and Physiological Responses of Horses.. Animals (Basel) 2020 Jan 17;10(1).
    doi: 10.3390/ani10010160pmc: PMC7022509pubmed: 31963529google scholar: lookup
  10. Ferlazzo A, Fazio E, Medica P. Behavioral features and effects of transport procedures on endocrine variables of horses. J Vet Behav 2020;39:21–31.
  11. Roy RC, Riley CB, Stryhn H, Dohoo I, Cockram MS. Infrared Thermography for the Ante Mortem Detection of Bruising in Horses Following Transport to a Slaughter Plant.. Front Vet Sci 2018;5:344.
    doi: 10.3389/fvets.2018.00344pmc: PMC6344440pubmed: 30705888google scholar: lookup
  12. Padalino B. Effects of the different transport phases on equine health status, behavior, and welfare: A review. J Vet Behav 2015;10:272–282.
  13. Padalino B, Raidal SL, Hall E, Knight P, Celi P, Jeffcott L, Muscatello G. Risk factors in equine transport-related health problems: A survey of the Australian equine industry.. Equine Vet J 2017 Jul;49(4):507-511.
    doi: 10.1111/evj.12631pubmed: 27564584google scholar: lookup
  14. Padalino B, Rogers CW, Guiver D, Thompson KR, Riley CB. A Survey-Based Investigation of Human Factors Associated With Transport Related Injuries in Horses.. Front Vet Sci 2018;5:294.
    doi: 10.3389/fvets.2018.00294pmc: PMC6262080pubmed: 30525050google scholar: lookup
  15. Grandin T. Perspectives on transportation issues: The importance of having physically fit cattle and pigs. J Anim Sci 2001;79:E201.
  16. Cockram MS. Fitness of animals for transport to slaughter.. Can Vet J 2019 Apr;60(4):423-429.
    pmc: PMC6417610pubmed: 30992599
  17. Grandin T, McGee K, Lanier J. Survey of Trucking Practices and Injury to Slaughter Horses. 1999.
  18. Fazio E, Medica P, Cravana C, Ferlazzo A. Cortisol response to road transport stress in calm and nervous stallions. J Vet Behav Clin Appl Res 2013;8:231–237.
  19. Schmidt A, Hödl S, Möstl E, Aurich J, Müller J, Aurich C. Cortisol release, heart rate, and heart rate variability in transport-naive horses during repeated road transport.. Domest Anim Endocrinol 2010 Oct;39(3):205-13.
  20. Padalino B, Henshall C, Raidal SL, Knight P, Celi P, Jeffcott L, Muscatello G. Investigations Into Equine Transport-Related Problem Behaviors: Survey Results. J Equine Vet Sci 2017;48:166–173.e2.
  21. Marsbøll AF, Christensen JW. Effects of handling on fear reactions in young Icelandic horses.. Equine Vet J 2015 Sep;47(5):615-9.
    doi: 10.1111/evj.12338pubmed: 25138590google scholar: lookup
  22. Sankey C, Henry S, Clouard C, Richard-Yris MA, Hausberger M. Asymmetry of behavioral responses to a human approach in young naive vs. trained horses.. Physiol Behav 2011 Sep 1;104(3):464-8.
    doi: 10.1016/j.physbeh.2011.05.009pubmed: 21605580google scholar: lookup
  23. Visser EK, van Reenen CG, van der Werf JT, Schilder MB, Knaap JH, Barneveld A, Blokhuis HJ. Heart rate and heart rate variability during a novel object test and a handling test in young horses.. Physiol Behav 2002 Jun 1;76(2):289-96.
    doi: 10.1016/S0031-9384(02)00698-4pubmed: 12044602google scholar: lookup
  24. Fazio E, Medica P, Cravana C, Ferlazzo AA. Pituitary-adrenocortical adjustments to transport stress in horses with previous different handling and transport conditions.. Vet World 2016 Aug;9(8):856-61.
  25. Sutton GA, Paltiel O, Soffer M, Turner D. Validation of two behaviour-based pain scales for horses with acute colic.. Vet J 2013 Sep;197(3):646-50.
    doi: 10.1016/j.tvjl.2013.04.007pubmed: 23993390google scholar: lookup
  26. Bonett DG. Sample size requirements for estimating intraclass correlations with desired precision.. Stat Med 2002 May 15;21(9):1331-5.
    doi: 10.1002/sim.1108pubmed: 12111881google scholar: lookup
  27. Wulf M, Aurich J, May AC, Aurich C. Sex differences in the response of yearling horses tohandling by unfamiliar humans. J Vet Behav Clin Appl Res 2013;8:238–244.
  28. ISES Principles of Learning Theory in Equitation. [(accessed on 4 June 2021)]; Available online: https://equitationscience.com/learning-theory/
  29. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements.. Phys Ther 2005 Mar;85(3):257-68.
    doi: 10.1093/ptj/85.3.257pubmed: 15733050google scholar: lookup
  30. Council of the European Union. Council Regulation (EC) No. 1/2005 of 22 December 2004 on the Protection of Animals during Transport and Related Operations and Amending Directives 64/432/EEC and 93/119/EC and Regulation (EC) No. 1255/97. 2005.
  31. Dalla Costa E, Dai F, Murray LAM, Guazzetti S, Canali E, Minero M. A study on validity and reliability of on-farm tests to measure human-animal relationship in horses and donkeys. Appl Anim Behav Sci 2015;163:110–121.
  32. Padalino B, Raidal SL, Carter N, Celi P, Muscatello G, Jeffcott L, de Silva K. Immunological, clinical, haematological and oxidative responses to long distance transportation in horses.. Res Vet Sci 2017 Dec;115:78-87.
    doi: 10.1016/j.rvsc.2017.01.024pubmed: 28160731google scholar: lookup
  33. Bartolomé E, Sánchez MJ, Molina A, Schaefer AL, Cervantes I, Valera M. Using eye temperature and heart rate for stress assessment in young horses competing in jumping competitions and its possible influence on sport performance.. Animal 2013 Dec;7(12):2044-53.
    doi: 10.1017/S1751731113001626pubmed: 24067493google scholar: lookup
  34. Jansson A, Lindgren G, Velie BD, Solé M. An investigation into factors influencing basal eye temperature in the domestic horse (Equus caballus) when measured using infrared thermography in field conditions.. Physiol Behav 2021 Jan 1;228:113218.
    doi: 10.1016/j.physbeh.2020.113218pubmed: 33131744google scholar: lookup
  35. Redaelli V, Luzi F, Mazzola S, Bariffi GD, Zappaterra M, Nanni Costa L, Padalino B. The Use of Infrared Thermography (IRT) as Stress Indicator in Horses Trained for Endurance: A Pilot Study.. Animals (Basel) 2019 Mar 7;9(3).
    doi: 10.3390/ani9030084pmc: PMC6466296pubmed: 30866503google scholar: lookup
  36. Meagher RK. Observer ratings: Validity and value as a tool for animal welfare research. Appl Anim Behav Sci 2009;119:1–14.
  37. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer.. Front Public Health 2018;6:149.
    doi: 10.3389/fpubh.2018.00149pmc: PMC6004510pubmed: 29942800google scholar: lookup
  38. Silva NEOF, Trindade PHE, Oliveira AR, Taffarel MO, Moreira MAP, Denadai R, Rocha PB, Luna SPL. Validation of the Unesp-Botucatu composite scale to assess acute postoperative abdominal pain in sheep (USAPS).. PLoS One 2020;15(10):e0239622.
  39. Eisinga R, Grotenhuis Mt, Pelzer B. The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown?. Int J Public Health 2013 Aug;58(4):637-42.
    doi: 10.1007/s00038-012-0416-3pubmed: 23089674google scholar: lookup
  40. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971;76:378–382.
    doi: 10.1037/h0031619google scholar: lookup
  41. Fleiss JL. The Design and Analysis of Clinical Experiments. 1999; pp. 1–32.
    doi: 10.1002/9781118032923.ch1google scholar: lookup
  42. Cohen J. Statistical Power Analysis for the Behavioural Sciences. 1988.
  43. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.. J Chiropr Med 2016 Jun;15(2):155-63.
    doi: 10.1016/j.jcm.2016.02.012pmc: PMC4913118pubmed: 27330520google scholar: lookup
  44. Field A, Miles J, Field Z. Discovering Statistics Using SPSS. 2009.
  45. Greiner M, Pfeiffer D, Smith RD. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests.. Prev Vet Med 2000 May 30;45(1-2):23-41.
    doi: 10.1016/S0167-5877(00)00115-Xpubmed: 10802332google scholar: lookup
  46. Barile VL, Menchetti L, Casano AB, Brecchia G, Melo de Sousa N, Zelli R, Canali C, Beckers JF, Barbato O. Approaches to Identify Pregnancy Failure in Buffalo Cows.. Animals (Basel) 2021 Feb 12;11(2).
    doi: 10.3390/ani11020487pmc: PMC7917614pubmed: 33673362google scholar: lookup
  47. Hallgren KA. Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial.. Tutor Quant Methods Psychol 2012;8(1):23-34.
    doi: 10.20982/tqmp.08.1.p023pmc: PMC3402032pubmed: 22833776google scholar: lookup
  48. van Loon JPAM, Van Dierendonck MC. Objective pain assessment in horses (2014-2018).. Vet J 2018 Dec;242:1-7.
    doi: 10.1016/j.tvjl.2018.10.001pubmed: 30503538google scholar: lookup
  49. Czycholl I, Klingbeil P, Krieter J. Interobserver Reliability of the Animal Welfare Indicators Welfare Assessment Protocol for Horses.. J Equine Vet Sci 2019 Apr;75:112-121.
    doi: 10.1016/j.jevs.2019.02.005pubmed: 31002084google scholar: lookup
  50. Waiblinger S, Boivin X, Pedersen V, Tosi MV, Janczak AM, Visser EK, Jones RB. Assessing the human-animal relationship in farmed species: A critical review. Appl Anim Behav Sci 2006;101:185–242.
  51. Windschnurer I, Schmied C, Boivin X, Waiblinger S. Reliability and inter-test relationship of tests for on-farm assessment of dairy cows’ relationship to humans. Appl Anim Behav Sci 2008;114:37–53.
  52. Czycholl I, Menke S, Straßburg C, Krieter J. Reliability of different behavioural tests for growing pigs on-farm. Appl Anim Behav Sci 2019;213:65–73.
  53. Czycholl I, Büttner K, Klingbeil P, Krieter J. An Indication of Reliability of the Two-Level Approach of the AWIN Welfare Assessment Protocol for Horses.. Animals (Basel) 2018 Jan 5;8(1).
    doi: 10.3390/ani8010007pmc: PMC5789302pubmed: 29303962google scholar: lookup
  54. Minero M, Dalla Costa E, Dai F, Canali E, Barbieri S, Zanella A, Pascuzzo R, Wemelsfelder F. Using qualitative behaviour assessment (QBA) to explore the emotional state of horses and its association with human-animal relationship. Appl Anim Behav Sci 2018;204:53–59.
  55. Bussières G, Jacques C, Lainay O, Beauchamp G, Leblond A, Cadoré JL, Desmaizières LM, Cuvelliez SG, Troncy E. Development of a composite orthopaedic pain scale in horses.. Res Vet Sci 2008 Oct;85(2):294-306.
    doi: 10.1016/j.rvsc.2007.10.011pubmed: 18061637google scholar: lookup
  56. McCall CA, Hall S, McElhenney WH, Cummins KA. Evaluation and comparison of four methods of ranking horses based on reactivity. Appl Anim Behav Sci 2006;96:115–127.
  57. Janczarek I, Kedzierski W. Emotional Response to Novelty and to Expectation of Novelty in Young Race Horses. J Equine Vet Sci 2011;31:549–554.
  58. Søndergaard E, Halekoh U. Young horses’ reactions to humans in relation to handling and social environment. Appl Anim Behav Sci 2003;84:265–280.
  59. Jezierski T, Jaworski Z, Górecka A. Effects of handling on behaviour and heart rate in Konik horses: Comparison of stable and forest reared youngstock. Appl Anim Behav Sci 1999;62:1–11.
  60. Rankins EM, Wickens CL. A systematic review of equine personality. Appl Anim Behav Sci 2020;231:105076.
  61. Fureix C, Pagès M, Bon R, Lassalle JM, Kuntz P, Gonzalez G. A preliminary study of the effects of handling type on horses' emotional reactivity and the human-horse relationship.. Behav Processes 2009 Oct;82(2):202-10.
    doi: 10.1016/j.beproc.2009.06.012pubmed: 19591910google scholar: lookup
  62. Padalino B, Raidal SL, Knight P, Celi P, Jeffcott L, Muscatello G. Behaviour during transportation predicts stress response and lower airway contamination in horses.. PLoS One 2018;13(3):e0194272.

Citations

This article has been cited 8 times.
  1. Felici M, Nanni Costa L, Zappaterra M, Bozzo G, Di Pinto P, Minero M, Padalino B. Journeys, Journey Conditions, and Welfare Assessment of Broken (Handled) Horses on Arrival at Italian Slaughterhouses. Animals (Basel) 2022 Nov 12;12(22).
    doi: 10.3390/ani12223122pubmed: 36428350google scholar: lookup
  2. Riva MG, Sobrero L, Menchetti L, Minero M, Padalino B, Dalla Costa E. Unhandled horses classified with broken/unbroken test (BUT) exhibit longer avoidance, flight reactions, and displacement behaviors when approached by humans. Front Vet Sci 2022;9:1022255.
    doi: 10.3389/fvets.2022.1022255pubmed: 36225797google scholar: lookup
  3. Nielsen SS, Alvarez J, Bicout DJ, Calistri P, Canali E, Drewe JA, Garin-Bastuji B, Gonzales Rojas JL, Gortázar Schmidt C, Michel V, Miranda Chueca MÁ, Padalino B, Pasquali P, Roberts HC, Spoolder H, Stahl K, Velarde A, Viltrop A, Winckler C, Earley B, Edwards S, Faucitano L, Marti S, Miranda de La Lama GC, Costa LN, Thomsen PT, Ashe S, Mur L, Van der Stede Y, Herskin M. Welfare of equidae during transport. EFSA J 2022 Sep;20(9):e07444.
    doi: 10.2903/j.efsa.2022.7444pubmed: 36092762google scholar: lookup
  4. Cartoni Mancinelli A, Mattioli S, Menchetti L, Dal Bosco A, Chiattelli D, Angelucci E, Castellini C. Validation of a behavior observation form for geese reared in agroforestry systems. Sci Rep 2022 Sep 7;12(1):15152.
    doi: 10.1038/s41598-022-18070-6pubmed: 36071073google scholar: lookup
  5. Zappaterra M, Nanni Costa L, Felici M, Minero M, Perniola F, Tullio D, Padalino B. Journeys, Journey Conditions, and Welfare Assessment of Unbroken (Unhandled) Horses on Arrival at a Slaughterhouse in Italy. Animals (Basel) 2022 Aug 15;12(16).
    doi: 10.3390/ani12162083pubmed: 36009673google scholar: lookup
  6. Menchetti L, Nanni Costa L, Zappaterra M, Padalino B. Effects of Reduced Space Allowance and Heat Stress on Behavior and Eye Temperature in Unweaned Lambs: A Pilot Study. Animals (Basel) 2021 Dec 5;11(12).
    doi: 10.3390/ani11123464pubmed: 34944241google scholar: lookup
  7. Loftus L, Newman A, Leach M, Asher L. Exploring the induction and measurement of positive affective state in equines through a personality-centred lens. Sci Rep 2025 May 27;15(1):18550.
    doi: 10.1038/s41598-025-98034-8pubmed: 40425817google scholar: lookup
  8. Cartoni Mancinelli A, Trocino A, Menchetti L, Chiattelli D, Ciarelli C, Castellini C. New approaches to selecting a scan-sampling method for chicken behavioral observations and their practical implications. Sci Rep 2023 Oct 11;13(1):17177.
    doi: 10.1038/s41598-023-44126-2pubmed: 37821498google scholar: lookup