There is growing interest in relating taste perception to diet and healthy aging. However, there is still limited information on the influence of age, sex and genetics on taste acuity as well as on the relationship between taste perception and taste preferences. We have analysed the influence of age on the intensity rating of the five basic tastes: sweet, salty, bitter, sour and umami (separately and jointly in a “total taste score”) and their modulation by sex and genetics in a relatively healthy population (men and women) aged 18–80 years (n = 1020 Caucasian European participants). Taste perception was determined by challenging subjects with solutions of the five basic tastes using standard prototypical tastants (6-n-propylthiouracil (PROP), NaCl, sucrose, monopotassium glutamate and citric acid) at 5 increasing concentrations (I to V). We also measured taste preferences and determined the polymorphisms of the genes taste 2 receptor member 38 (TAS2R38), taste 1 receptor member 2 (TAS2R38) and sodium channel epithelial 1 beta subunit (SCNN1B), as TAS2R38-rs713598, TAS1R2-rs35874116 and SCNN1B-rs239345 respectively. We found a statistically significant decrease in taste perception (“total taste score”) with increasing age for all the concentrations analysed. This association was stronger for the higher concentrations (p = 0.028; p = 0.012; p = 0.005; p = 4.20 × 10−5 and p = 1.48 × 10−7, for I to V in the multivariable-adjusted models). When we analysed taste qualities (using concentration V), the intensity rating of all the 5 tastes was diminished with age (p < 0.05 for all). This inverse association differed depending on the test quality, being higher for bitter (PROP) and sour. Women perceived taste significantly more intense than men (p = 1.4 × 10−8 for total taste score). However, there were differences depending on the taste, umami being the lowest (p = 0.069). There was a complex association between the ability to perceive a taste and the preference for the same. Significant associations were, nevertheless, found between a higher perception of sour taste and a higher preference for it in women. In contrast, the higher perception of sweet was significantly associated with a higher preference for bitter in both, men and women. The TAS2R38-rs713598 was strongly associated with bitter (PROP) taste (p = 1.38 × 10−50), having a significant interaction with sex (p = 0.030). The TAS1R2-rs35874116 was not significantly associated with sweet, whereas the SCNN1B-rs239345 was associated (p = 0.040) with salty taste. In conclusion, the inverse association between age and perceived taste intensity as well as the additional influence of sex and some genetic polymorphisms give rise to large inter-individual differences in taste perception and taste preferences that should be taken into account in future studies and for applications in precision nutrition for healthy aging.