(1) Background: The relatively high rate of contralateral prophylactic mastectomy (CPM) among women with early stage unilateral breast cancer (BC) has raised concerns. We sought to assess the influence of partners, physicians, and the media on the decision of women with unilateral BC to undergo CPM and identify clinicopathological variables associated with the decision to undergo CPM. (2) Patients and Methods: Women with stage 0 to III unilateral BC who underwent CPM between January 2010 and December 2017. Patients were surveyed regarding factors influencing their self-determined decision to undergo CPM. Partner, physician, and media influence factors were modeled by logistic regressions with adjustments for a family history of breast cancer and pathological stage. (3) Results: 397 (29.6%) patients completed the survey and were included in the study. Partners, physicians, and the media significantly influenced patients’ decision to undergo CPM. The logistic regression models showed that, compared to self-determination alone, overall influence on the CPM decision was significantly higher for physicians (p = 0.0006) and significantly lower for partners and the media (p < 0.0001 for both). Fifty-nine percent of patients’ decisions were influenced by physicians, 28% were influenced by partners, and only 17% were influenced by the media. The model also showed that patients with a family history of BC had significantly higher odds of being influenced by a partner than did those without a family history of BC (p = 0.015). (4) Conclusions: Compared to self-determination, physicians had a greater influence and partners and the media had a lower influence on the decision of women with unilateral BC to undergo CPM. Strong family history was significantly associated with a patient’s decision to undergo CPM.