Breast malignancy is the second most common cause of cancer death in women. However, less common breast masses can mimic carcinoma and can pose diagnostic challenges. This case-based review describes a spectrum of rare breast neoplastic and non-neoplastic masses ranging from malignant to benign entities. Malignant masses in this review include adenoid cystic carcinoma, spindle cell lipoma, granular cell tumor, angiosarcoma, glomus tumor, adenosquamous carcinoma, and myofibroblastoma. Benign masses include sarcoidosis, diabetic mastopathy, and cat scratch disease. Demographics and, when relevant, clinical presentation are summarized. Breast imaging appearance on mammography and ultrasound are highlighted along with radiology-pathology correlation with the appearance and characteristics of the histopathological specimen of these rare masses.