The effects of solar activity and the accompanying space weather events on human pathological conditions, physiological parameters and other psycho-physiological disturbances have been analyzed in numerous recent investigations. Moreover, many of these studies have particularly focused on the different physical reactions humans have, according to their sex, during variations in the physical environment. In the framework of the above, this work analyses heart rate data obtained from volunteers (687 men and 534 women) from three different regions (Athens, Piraeus and Heraklion) of Greece in relation to the geophysical activity and variations of environmental factors. Dst index and Ap index data, along with cosmic ray intensity data derived from the Athens Neutron Monitor Station (A.Ne.Mo.S.), were used. The study expands from April 2011 to January 2018, covering solar cycle 24. The ANalysis Of Variance (ANOVA) and the superimposed epochs methods were used in order to examine heart rate variations depending on sex. Results revealed that women tend to be more sensitive to physical environmental changes. Statistically significant results are related to the geomagnetic activity but were not obtained for cosmic ray variations.