Grey matter volume (GMV) may be associated with polygenic risk for schizophrenia (PRS-SZ) and severe cognitive deficits in people with schizophrenia, schizoaffective disorder (collectively SSD), and bipolar disorder (BD). This study examined the interactive effects of PRS-SZ and cognitive subtypes of SSD and BD in relation to GMV. Two-step cluster analysis was performed on 146 clinical cases (69 SSD and 77 BD) assessed on eight cognitive domains (verbal and visual memory, executive function, processing speed, visual processing, language ability, working memory, and planning). Among them, 55 BD, 51 SSD, and 58 healthy controls (HC), contributed to focal analyses of the relationships between cognitive subtypes, PRS-SZ and their interaction on GMV. Two distinct cognitive subtypes were evident among the combined sample of cases: a ‘cognitive deficit’ group (CD; N = 31, 20SSD/11BD) showed severe impairment across all cognitive indices, and a ‘cognitively spared’ (CS; N = 75; 31SSD/44BD) group showed intermediate cognitive performance that was significantly worse than the HC group but better than the CD subgroup. A cognitive subgroup-by-PRS-SZ interaction was significantly associated with GMV in the left precentral gyrus. Moderation analyses revealed a significant negative relationship between PRS-SZ and GMV in the CD group only. At low and average (but not high) PRS-SZ, larger precentral GMV was evident in the CD group compared to both CS and HC groups, and in the CS group compared to HCs. This study provides evidence for a relationship between regional GMV changes and PRS-SZ in psychosis spectrum cases with cognitive deficits, but not in cases cognitively spared.