Virtual agents for studying the effect of emotional feedback on motor learning: A feasibility study

Feedback is an essential component of training focused on acquisition or modification of motor skills, i.e. motor learning. External feedback augments task performance information, can be provided by a human instructor/therapist or a digital agent and can impact motor learning. External feedback is often accompanied by emotional content (e.g., the tone of voice or intonation could be negative or positive). The ability to process emotional feedback and its effect on learning may vary depending on the learner’s personal characteristics (personality, emotional regulation). This work evaluated the effect of emotional feedback from a virtual agent on learning of a bimanual motor task. Sixty women practiced a virtual archery task while receiving either positive (n=31) or negative (n=29) emotional feedback from a virtual robot for 5 minutes. Movement accuracy and kinematics were measured using a head-mounted display (HTC Vive) and task engagement using electroencephalography. Training improved shot accuracy and movement characteristics in both groups. Positive emotional feedback (e.g., “good job!”) was associated with longer bow-pulling during and after training, and lower task engagement after training, compared with negative feedback (e.g., “I’m disappointed”). Finally, participants with higher trait negative affect tended to improve more under negative feedback, while those with a higher tendency for cognitive reappraisal improved less. These results suggest that virtual agents can harness the social influence of emotions and provide emotional feedback for motor learning. The effects of feedback valence and learners’ characteristics need to be considered when designing digital environments for motor rehabilitation.