Phyllodes tumors are rare fibro-epithelial tumors that are known to grow very rapidly in a short period. The tumor is known for its propensity to recur locally despite adequate wide excisions and even mastectomies. This case report is of a woman who had a recurrent Phyllodes tumor in a peculiar location. Phyllodes tumours are generally confined to arising from the breast; however, in this case, we found it arising from the anterior abdominal wall. The tumour was removed with a 2 cm margin of excision. The large defect that remained after excision was reconstructed using a latissimus dorsi flap, along with a split skin graft. The immediate post-operative period was uneventful. Ultrasonography did not detect a recurrence after 12 months. We presented relevant literature highlighting tumor infiltration into the anterior abdominal wall, our surgical management and reconstructive planning for closure of a large post-excision defect. In order to prevent a recurrence of phyllodes tumour, it is important to excise the tumour with wide margins of 2 cm. A multi-disciplinary team approach is essential for the reconstruction of the remaining defect.