Accessory breast tissue is the superfluous tissue that persists from normal embryological development. The pathological involvement of this tissue is quite uncommon and can be diagnostically challenging. Phyllodes tumor is a rare mammary fibroepithelial lesion that has been seldom reported to originate from the accessory breast tissue. This article describes an extremely rare case of a benign phyllodes tumor arising from the accessory breast tissue of the left axilla in a 40-year-old female so as to create awareness about rare tumors occurring within rare anomalies.