Background: Computer-aided methods have been widely applied to diagnose lesions detected on breast MRI, but fullyautomatic diagnosis using deep learning is rarely reported. Purpose: To evaluate the diagnostic accuracy of mass lesions using region of interest (ROI)-based, radiomics and deeplearning methods, by taking peritumor tissues into consideration. Study Type: Retrospective. Population: In all, 133 patients with histologically confirmed 91 malignant and 62 benign mass lesions for training (74 patients with 48 malignant and 26 benign lesions for testing). Field Strength/Sequence: 3T, using the volume imaging for breast assessment (VIBRANT) dynamic contrast-enhanced (DCE) sequence. Assessment: 3D tumor segmentation was done automatically by using fuzzy-C-means algorithm with connectedcomponent labeling. A total of 99 texture and histogram parameters were calculated for each case, and 15 were selected using random forest to build a radiomics model. Deep learning was implemented using ResNet50, evaluated with 10-fold crossvalidation. The tumor alone, smallest bounding box, and 1.2, 1.5, 2.0 times enlarged boxes were used as inputs. Statistical Tests: The malignancy probability was calculated using each model, and the threshold of 0.5 was used to make a diagnosis. Results: In the training dataset, the diagnostic accuracy was 76% using three ROI-based parameters, 84% using the radiomics model, and 86% using ROI + radiomics model. In deep learning using the per-slice basis, the area under the receiver operating characteristic (ROC) was comparable for tumor alone, smallest and 1.2 times box (AUC = 0.97-0.99), which were significantly higher than 1.5 and 2.0 times box (AUC = 0.86 and 0.71, respectively). For per-lesion diagnosis, the highest accuracy of 91% was achieved when using the smallest bounding box, and that decreased to 84% for tumor alone and 1.2 times box, and further to 73% for 1.5 times box and 69% for 2.0 times box. In the independent testing dataset, the perlesion diagnostic accuracy was also the highest when using the smallest bounding box, 89%. Data Conclusion: Deep learning using ResNet50 achieved a high diagnostic accuracy. Using the smallest bounding box containing proximal peritumor tissue as input had higher accuracy compared to using tumor alone or larger boxes. Level of Evidence: 3 Technical Efficacy: Stage 2