Inter- and intra-rater reliability of linear scoring in warmblood breeding.

Inter-rater and intra-rater reliability of linear scoring for conformation traits in German Warmblood horses was assessed, revealing moderate agreement for some traits like head shape but low reliability for others such as shoulder length, indicating a need for improved methods in phenotypic evaluation.

Objective and Importance of Study

• Reliable phenotypic measurements are crucial for effective animal breeding programs and genetic studies.
• Subjective assessment can bias heritability estimates, impacting the power of genome-based analyses.
• Warmblood horses are typically evaluated for conformation traits using a linear scoring system, but the reliability of this method had not been adequately evaluated previously.
• The study aimed to quantify both inter-rater (between different experts) and intra-rater (same expert over time) reliability of linear scoring of conformation traits in German Warmblood stallions.

Methods

• Participants: 11 professional equine breeding experts, including breeding judges, participated in the study.
• Challenges: They scored 17 linear conformation traits using standardized side-view images of 34 Warmblood stallions.
• Image Source: Stallion images were archival photographs from the Hanoverian State Stud Celle, covering birth years from 1929 to 1993.
• Scoring Scale: Traits were scored on a linear scale from -3 to +3, characterizing variation around a breed standard.
• Repeatability: Participants repeated the scoring after approximately one month to assess intra-rater reliability.
• Statistical Analysis: Reliability was evaluated using intraclass correlation coefficients (ICCs), which measure consistency and agreement among raters.

Key Findings

• Mean scores across traits were near zero, indicating that raters generally scored traits around the average phenotype.
• Low average absolute deviations suggested cautious use of the full scale range (-3 to +3), possibly indicating a conservative scoring style.
• Inter-rater reliability (agreement between different experts) was generally lower than intra-rater reliability (consistency of the same expert over time).
• Specific traits showed markedly different reliability patterns:
  • Head shape had moderate to good reliability:
    • Inter-rater reliability ICCs were approximately 0.50 (moderate agreement).
    • Intra-rater ICC was higher (~0.71 ± 0.23), indicating individual judges were relatively consistent over time.
  • Traits like Shoulder Length and Carpal Joints had ICCs near zero, indicating poor reliability and disagreement among raters.

Conclusion and Implications

• The variable reliability across traits suggests that the current linear scoring approach may not consistently produce reliable phenotypic data for all conformation traits.
• Lower inter-rater reliability indicates that subjective judgment leads to disagreement between experts, which can undermine genetic evaluations and selection decisions.
• The results highlight a potential need for:
  • Better standardization of scoring protocols to reduce variation.
  • Training programs to improve scoring consistency among judges.
  • Development of alternative, possibly more objective phenotyping methods (e.g., automated measurements, 3D imaging) for assessing equine conformation.
• Overall, ensuring reliable phenotype data will improve the accuracy of genetic analyses and thus enhance breeding program outcomes.