Introduction: Anxiety disorders manifest in women more than in men by almost twofold. This narrative review aims to summarize the sex-related biological factors, which underpin anxiety, focusing on the interactions of sex and tryptophan/serotonin with anxiety. Methods: A literature search was conducted using Google Scholar, PubMed/MEDLINE, Scopus, and EMBASE databases from inception until December 31, 2017. Results: This review shows that sex may interact with many serotonin functions thereby modulating anxiety, including 5-HT1A and 5-HT2C receptors, 5-HT transporter and central 5-HT concentrations and metabolism. Sex-steroids modulate the expression of serotonin transporter genes, creating a difference in serotonin availability. Sex and estrous cycle phases lead to varying anxiety responses to tryptophan depletion. Testosterone, progesterone and estrogen are important factors in mediating sex differences in serotonin responses to anxietygenerating behavioral tests. At prenatal levels, there are sex-related differences in the reciprocal relationships between serotonin and the HPA-axis, which modulate anxiety-like behaviors. Activated immune-inflammatory pathways induce indoleamine-2,3-dioxynease (IDO) and the tryptophan catabolite (TRYCAT) pathway thereby increasing tryptophan degradation and increasing the production of TRYCATs including kynurenine and quinolinic acid, which may create an overall anxiogenic effect. The effects of immune activation on IDO are significantly more pronounced in women than men and therefore females may show increased levels of anxiogenic TRYCAT following immune challenge. Aberrations in the IDO-activated TRYCAT pathway are found in pregnant females and parturients and are associated with increased anxiety levels in the postnatal period.