Background/Objectives: Although manual therapies can be used for pain alleviation in fibromyalgia, there is no clear evidence about the processing of gentle, affective touch in this clinical condition. In fact, persistent painful sensations and psychological factors may impact the hedonic experience of touch. Methods: This observational cross-sectional study compared the subjective experience of affective touch between 14 women with fibromyalgia (age range: 35–70; range of years of education: 5–13) and 14 pain-free women (age range: 18–30; range of years of education: 13–19). The participants rated the pleasantness of slow and fast touches delivered by a brush, the experimenter’s hand, and a plastic stick. Tactile stimuli were either imagined or real to disentangle the contribution of top-down and bottom-up sensory components. Additionally, a self-report questionnaire explored the lifetime experiences of affective touch. Results: Akin to healthy counterparts, individuals with fibromyalgia rated slow touches delivered by the experimenter’s hand or a brush as more pleasant than fast touches, regardless of whether they were imagined or real. However, the intensity of pain affects only the imagined pleasantness in our participants with fibromyalgia. Furthermore, despite the fibromyalgia patients reporting fewer experiences of affective touch in childhood and adolescence, this evidence was not associated with the experimental outcomes. Conclusions: The hedonic experience of affective touch seems preserved in fibromyalgia despite poor intimate bodily contact in youth. We confirmed that bottom-up and top-down factors contribute to the affective touch perception in fibromyalgia: bodily pain may impact even more the expected pleasure than the actual experience. Future investigations may introduce neurophysiological measures of the implicit autonomic responses to affective touch in fibromyalgia. To conclude, although preliminary, our evidence may be in favor of manual therapies for pain relief in fibromyalgia.