We investigated gene–environment effects on structural brain endophenotype in bipolar disorder (BD) using a novel method of combining polygenic risk scores with epigenetic signatures since traditional methods of examining the family history and trauma effects have significant limitations. The study enrolled 119 subjects, including 55 BD spectrum (BDS) subjects diagnosed with BD or major depressive disorder (MDD) with subthreshold BD symptoms and 64 non-BDS subjects comprising 32 MDD subjects without BD symptoms and 32 healthy subjects. The blood samples underwent genome-wide genotyping and methylation quantification. We derived polygenic risk score (PRS) and methylation profile score (MPS) as weighted summations of risk single nucleotide polymorphisms and methylation probes, respectively, which were considered as molecular measures of genetic and environmental risks for BD. Linear regression was used to relate PRS, MPS, and their interaction to 44 brain structure measures quantified from magnetic resonance imaging (MRI) on 47 BDS subjects, and the results were compared with those based on family history and childhood trauma. After multiplicity corrections using false discovery rate (FDR), MPS was found to be negatively associated with the volume of the medial geniculate thalamus (FDR = 0.059, partial R2 = 0.208). Family history, trauma scale, and PRS were not associated with any brain measures. PRS and MPS show significant interactions on whole putamen (FDR = 0.09, partial R2 = 0.337). No significant gene–environment interactions were identified for the family history and trauma scale. PRS and MPS generally explained greater proportions of variances of the brain measures (range of partial R2 = [0.008, 0.337]) than the clinical risk factors (range = [0.004, 0.228]).