(1) Background: Patient sex is associated with differential outcome of many procedures although the exact mechanisms remain unknown. Especially in transplant surgery, surgeon-patient sex-concordance is rarely present for female patients and outcome may be negatively affected. (2) Methods: In this single-center retrospective cohort study, recipient, donor, and surgeon sex were evaluated and short- and long-term outcome was analyzed with regards to sex and sex-concordance of patients, donors, and surgeons. (3) Results: We included 425 recipients in our study; 50.1% of organ donors, 32.7% of recipients, and 13.9% of surgeons were female. Recipient-donor sex concordance was present in 82.7% of female recipients and in 65.7% of male recipients (p = 0.0002). Recipient-surgeon sex concordance was present in 11.5% of female recipients and in 85.0% of male recipients (p < 0.0001). Five-year patient survival was comparable between female and male recipients (70.0% vs. 73.3%, p = 0.3978). Five-year patient survival of female recipients treated by female surgeons was improved without reaching significance (81.3% vs. 68.4%, p = 0.3621). (4) Conclusions: Female recipients and female surgeons are underrepresented in liver transplant surgery. Societal factors influencing outcome of female patients suffering from end-stage organ failure need to be further examined and acted upon to possibly improve the outcome of female liver transplant recipients.