Estrogen receptor alpha (ER‐α) action is critical for hormone‐dependent breast cancer, and ER‐α dysregulation can lead to the emergence of resistance to endocrine therapy. Here, it is found that TRIM4 is downregulated in tamoxifen (TAM)‐resistant breast cancer cells, while the loss of TRIM4 is associated with an unfavorable prognosis. In vitro and in vivo experiments confirm that TRIM4 increased ER‐α expression and the sensitivity of breast cancer cells to TAM. Mechanistically, TRIM4 is found to target SET, and TRIM4‐SET interactions are mediated by the RING and B‐box domains of TRIM4 and the carboxyl terminus of SET. Moreover, it is determined that TRIM4 catalyzed the K48‐linked polyubiquitination of SET (K150 and K172), promoting its proteasomal degradation and disassociation from p53 and PP2A. Once released, p53 and PP2A are able to further promote ESR1 gene transcription and enhance mRNA stability. Moreover, univariate and multivariate Cox proportional hazards regression analyses confirm that TRIM4 expression is an independent predictor of overall survival and recurrence‐free survival outcomes in patients with ER‐α positive breast cancer. Taken together, the data highlights a previously undiscovered mechanism and suggest that TRIM4 is a valuable biomarker that can be analyzed to predict response to endocrine therapy in breast cancer patients.