A human–animal relationship can be developed through subsequent interactions, affected by the positive or negative emotional valence of the proceeding one. Horses implement a process of categorization to classify humans with whom they interact as positive, negative, or neutral stimuli by evaluating the kind of approach and the nature of the contact. In these terms, human–animal interactions are emotionally charged events, eliciting specific emotional states in both subjects involved. Although the human–horse relationship has been mainly investigated through behavioral analysis, physiological indicators are needed for a more objective assessment of the emotional responses. Heart rate variability (HRV) is a commonly used autonomic nervous system (ANS) correlate estimating the sympathovagal balance as a psychophysiological marker of emotion regulation in horses. We have assumed that long-term positive relationships with humans may have a positive and immediate impact on the emotional arousal of the horse, detectable, via ANS activity, during the interaction. We analyzed horses' heartbeat dynamics during their interaction with either familiar or unfamiliar handlers, applying a standardized experimental protocol consisting of three different conditions shifting from the absence of interaction to physical contact. The ANS signals were monitored through an innovative non-invasive wearable system, not interfering with the unconscious emotional response of the animal. We demonstrated that horses appeared to feel more relaxed while physically interacting (e.g., grooming on the right side) with some familiar handlers compared to the same task performed by someone unfamiliar. The shift of the sympathovagal balance toward a vagal predominance suggests that the horses experienced a decrease in stress response as a function not only of the handler's familiarity but also of the type of interaction they are experiencing. These results constitute the objective evidence of horses' capacity to individually recognize a familiar person, adding the crucial role of familiarity with the handler as a paramount component of human–animal interaction. Our rigorous methodological approach may provide a significant contribution to various fields such as animal welfare while further investigating the emotional side of the human–animal relationships.