ObjectivesThis study aims to build radiomics model of Breast Imaging Reporting and Data System (BI-RADS) category 4 and 5 mammographic masses extracted from digital mammography (DM) for mammographic masses characterization by using a sensitivity threshold similar to that of biopsy.Materials and MethodsThis retrospective study included 288 female patients (age, 52.41 ± 10.31) who had BI-RADS category 4 or 5 mammographic masses with an indication for biopsy. The patients were divided into two temporal set (training set, 82 malignancies and 110 benign lesions; independent test set, 48 malignancies and 48 benign lesions). A total of 188 radiomics features were extracted from mammographic masses on the combination of craniocaudal (CC) position images and mediolateral oblique (MLO) position images. For the training set, Pearson’s correlation and the least absolute shrinkage and selection operator (LASSO) were used to select non-redundant radiomics features and useful radiomics features, respectively, and support vector machine (SVM) was applied to construct a radiomics model. The receiver operating characteristic curve (ROC) analysis was used to evaluate the classification performance of the radiomics model and to determine a threshold value with a sensitivity higher than 98% to predict the mammographic masses malignancy. For independent test set, identical threshold value was used to validate the classification performance of the radiomics model. The stability of the radiomics model was evaluated by using a fivefold cross-validation method, and two breast radiologists assessed the diagnostic agreement of the radiomics model.ResultsIn the training set, the radiomics model obtained an area under the receiver operating characteristic curve (AUC) of 0.934 [95% confidence intervals (95% CI), 0.898–0.971], a sensitivity of 98.8% (81/82), a threshold of 0.22, and a specificity of 60% (66/110). In the test set, the radiomics model obtained an AUC of 0.901 (95% CI, 0.835–0.961), a sensitivity of 95.8% (46/48), and a specificity of 66.7% (32/48). The radiomics model had relatively stable sensitivities in fivefold cross-validation (training set, 97.39% ± 3.9%; test set, 98.7% ± 4%).ConclusionThe radiomics method based on DM may help reduce the temporarily unnecessary invasive biopsies for benign mammographic masses over-classified in BI-RADS category 4 and 5 while providing similar diagnostic performance for malignant mammographic masses as biopsies.