The present study builds on our prior work that demonstrated an association between pharmacogenetic interactions and 90-day readmission. In a substantially larger, more diverse study population of 19,999 adults tracked from 2010 through 2020 who underwent testing with a 13-gene pharmacogenetic panel, we included additional covariates to evaluate aggregate contribution of social determinants and medical comorbidity with the presence of identified gene-x-drug interactions to moderate 90-day hospital readmission (primary outcome). Univariate logistic regression analyses demonstrated that strongest associations with 90 day hospital readmissions were the number of medications prescribed within 30 days of a first hospital admission that had Clinical Pharmacogenomics Implementation Consortium (CPIC) guidance (CPIC medications) (5+ CPIC medications, odds ratio (OR) = 7.66, 95% confidence interval 5.45–10.77) (p < 0.0001), major comorbidities (5+ comorbidities, OR 3.36, 2.61–4.32) (p < 0.0001), age (65 + years, OR = 2.35, 1.77–3.12) (p < 0.0001), unemployment (OR = 2.19, 1.88–2.64) (p < 0.0001), Black/African-American race (OR 2.12, 1.47–3.07) (p < 0.0001), median household income (OR = 1.63, 1.03–2.58) (p = 0.035), male gender (OR = 1.47, 1.21–1.80) (p = 0.0001), and one or more gene-x-drug interaction (defined as a prescribed CPIC medication for a patient with a corresponding actionable pharmacogenetic variant) (OR = 1.41, 1.18–1.70). Health insurance was not associated with risk of 90-day readmission. Race, income, employment status, and gene-x-drug interactions were robust in a multivariable logistic regression model. The odds of 90-day readmission for patients with one or more identified gene-x-drug interactions after adjustment for these covariates was attenuated by 10% (OR = 1.31, 1.08–1.59) (p = 0.006). Although the interaction between race and gene-x-drug interactions was not statistically significant, White patients were more likely to have a gene-x-drug interaction (35.2%) than Black/African-American patients (25.9%) who were not readmitted (p < 0.0001). These results highlight the major contribution of social determinants and medical complexity to risk for hospital readmission, and that these determinants may modify the effect of gene-x-drug interactions on rehospitalization risk.