Multimodal Analysis of Child-Horse Therapeutic Interactions Using Vision-Language Models in an Australian Aboriginal Equine-Assisted Learning Program for Social and Emotional Well-Being: Development and Evaluation Study.

Overview

• This study developed and evaluated an AI-powered framework using vision-language models to analyze interactions between Aboriginal children and horses in a therapeutic equine-assisted learning program aimed at enhancing social and emotional well-being.
• The framework aims to improve the objectivity, consistency, and efficiency of documenting and assessing therapeutic activities compared to traditional practitioner observations.

Background and Motivation

• The human-horse bond has historically supported human development and is now applied in mental health promotion through equine-assisted learning (EAL) programs.
• Traditional documentation in EAL relies heavily on subjective practitioner notes, checklists, and interviews, which can be inconsistent, subjective, and time-consuming.
• There is a need for objective, standardized, and efficient methods to assess therapeutic interactions to better support mental health outcomes.
• This study focuses on an Australian Aboriginal youth program, Yawardani Jan-ga, located in the Kimberley region, which employs EAL to promote social and emotional well-being among children.

Objectives

• To develop an AI-powered digital health framework that objectively analyzes child-horse interactions during EAL sessions using vision-language models (VLMs).
• To evaluate the framework’s performance in classifying therapeutic activities, physical contact (proxemics), and body posture related to confidence states.
• To address challenges of subjectivity, time demands, and inconsistent documentation through automated analysis.

Methods and Framework Components

• The framework integrates three core modules:
• Activity Classification Module:
  • Utilizes a VLM called SigLIP2 (Sigmoid Loss for Language-Image Pre-training 2) with few-shot learning.
  • Classifies nine distinct therapeutic activities occurring between the child and horse.
  • Operates effectively even with limited labeled examples by leveraging few-shot learning.
• Proxemics Analysis Module:
  • Employs the Qwen2.5-VL-7B VLM to analyze physical contact or spatial relationships between child and horse.
  • Classifies varying degrees or types of physical interaction.
  • Aims to objectively assess closeness and tactile engagement important in therapy.
• Body Posture Analysis Module:
  • Uses multimodal chain-of-thought prompting and self-consistency methods to infer children’s body posture.
  • Classifies postures that indicate confidence-related emotional states.
  • Includes interpretable reasoning to provide transparent AI decision explanations.
• The framework was trained and validated on real-world images taken during actual therapeutic sessions with Aboriginal children participating in the Yawardani Jan-ga program.

Results

• Activity Classification Module:
  • Achieved 90.6% accuracy in identifying therapeutic activities.
  • Macroaveraged metrics: precision 92.0%, recall 91.1%, and F-score 91.1% under a 2-shot learning configuration.
• Proxemics Analysis Module:
  • Achieved 88.6% accuracy in classifying physical contact between child and horse.
  • Macroaveraged metrics: precision 89.0%, recall 88.0%, and F-score 88.5% with only 1-shot learning.
• Body Posture Analysis Module:
  • Assessed for internal reliability rather than direct classification accuracy.
  • Metrics reported include average consistency (0.916), agreement rate (0.705), rationale coherence (0.857), and vote entropy (0.174), indicating strong reliability and stable reasoning in classification.

Conclusions and Implications

• The study demonstrates that AI-driven vision-language models can effectively and objectively analyze crucial aspects of child-horse therapeutic interactions in EAL settings.
• The developed framework can complement practitioner observations by reducing administrative burden and increasing assessment consistency.
• Such objective analysis tools hold promise for strengthening evidence-based evaluation of mental health promotion programs, particularly those focused on social and emotional well-being in Indigenous youth.
• Implementation of these AI approaches may enable better documentation practices, facilitate evaluation, and contribute to the optimization of equine-assisted learning interventions.
• Future work could explore integration into routine clinical workflows and expansion to other populations and therapeutic contexts.