Background. Cognitive impairments are documented in schizophrenia (SZ) and bipolar disorder (BD) and may be related to gray matter volumes (GMVs). Thus, this study is aimed at exploring whether the association between cognitive impairments and GMV alterations is similar in patients with SZ and BD and understanding the underlying neurobiological mechanisms. Methods. A total of 137 adult subjects (46 with SZ, 35 with BD, and 56 age-, sex-, and education-matched healthy controls (HC)) completed the MATRICS Consensus Cognitive Battery (MCCB) and structural magnetic resonance imaging scanning. We performed group comparisons of the cognitive impairments, the GMV alterations, and the association between them. Results. Compared with HC, the patients with SZ and BD showed shared deficits in 4 cognitive domains (i.e., processing speed, working memory, problem solving, and social cognition) and the composite. SZ and BD had commonly decreased GMVs, mainly in the insula, superior temporal pole, amygdala, anterior cingulate, and frontal cortices (superior, middle, opercular inferior, and orbital frontal gyrus). No correlation between MCCB scores and GMVs was detected in SZ. However, for BD, working memory was relevant to the right hemisphere (i.e., right insula, amygdala, superior temporal pole, and medial and dorsolateral superior frontal gyrus). Limitations. The major limitations were that not all patients were the first-episode status and no medication. Conclusions. The association was mainly limited to the BD group. Thus, the underlying pathophysiology of the cognitive deficits, in terms of GMV alterations, may be diverse between two disorders.