Self-esteem is an important factor determining QoL after surgical procedures leading to bodily deformities associated with cancer treatment. However, there are few data on which components of self-esteem are most closely related to QoL. The article presents two studies that aim to fill this gap. Study 1 concerns changes in global self-esteem and QoL in patients treated surgically for oral cancer (n = 35); Study 2 concerns changes in explicit and implicit self-esteem and QoL in women with breast cancer undergoing mastectomy (n = 96). The study was longitudinal with two measurements: before and after surgery. Both studies used the EORTC QLQ-C30 and Rosenberg’s SES questionnaires. In Study 2, the Implicit Association Test (IAT) was additionally performed. The patients’ global QoL and self-esteem deteriorated after surgery. In Study 1, patients with higher initial self-esteem showed a greater range of decreased self-esteem and QoL than patients with initially low self-esteem. In Study 2, the largest decreases in various dimensions of QoL and explicit self-esteem were observed in women with fragile self-esteem. A group of women with high explicit and implicit self-esteem showed the best QoL after mastectomy. Cancer patients with high, fragile self-esteem are at risk of the greatest deterioration in QoL and self-image after cancer surgery. These people should be given special psycho-oncological care.