Small everyday gestures such as a tap on the shoulder can affect the way humans feel and act. Touch can have a calming effect and alter the way stress is handled, thereby promoting mental and physical health. Due to current technical advances and the growing role of intelligent robots in households and healthcare, recent research also addressed the potential of robotic touch for stress reduction. In addition, touch by non-human agents such as animals or inanimate objects may have a calming effect. This conceptual article will review a selection of the most relevant studies reporting the physiological, hormonal, neural, and subjective effects of touch on stress, arousal, and negative affect. Robotic systems capable of non-social touch will be assessed together with control strategies and sensor technologies. Parallels and differences of human-to-human touch and human-to-non-human touch will be discussed. We propose that, under appropriate conditions, touch can act as (social) signal for safety, even when the interaction partner is an animal or a machine. We will also outline potential directions for future research and clinical relevance. Thereby, this review can provide a foundation for further investigations into the beneficial contribution of touch by different agents to regulate negative affect and arousal in humans.