The aim of this study was to explore whether intravoxel incoherent motion (IVIM) combined with a dynamic contrast–enhanced magnetic resonance imaging (DCE–MRI) quantitative model can improve the ability to distinguish between benign and malignant BI-RADS 4 breast lesions. We enrolled 100 patients who underwent breast MRI at our institution and extracted the quantitative parameters of lesions with a post-processing workstation. Statistical differences in these parameters between benign and malignant BI-RADS 4 lesions were assessed using a two independent samples t-test or a Mann–Whitney U test. Binary logistic regression analysis was performed to establish five diagnostic models (model_ADC, model_IVIM, model_DCE, model_DCE+ADC, and model_DCE+IVIM). Receiver operating characteristic (ROC) curves, leave-one-out cross-validation, and the Delong test were used to assess and compare the diagnostic performance of these models. The model_DCE+IVIM showed the highest area under the curve (AUC) of 0.903 (95% confidence interval (CI): 0.828–0.953, sensitivity: 87.50%, specificity: 85.00%), which was significantly higher than that of model_ADC (p = 0.014) and model_IVIM (p = 0.033). The model_ADC had the lowest diagnostic performance (AUC = 0.768, 95%CI: 0.672–0.846) but was not significantly different from model_IVIM (p = 0.168). The united quantitative model with DCE–MRI and IVIM could improve the ability to evaluate the malignancy in BI-RADS 4 lesions, and unnecessary breast biopsies may be obviated.