Neoadjuvant therapy in breast cancer can downstage axillary lymph nodes and reduce extent of axillary surgery. As such, accurate determination of nodal status after neoadjuvant therapy and before surgery impacts surgical management. There are scarce data on the diagnostic accuracy of breast magnetic resonance imaging (MRI) for nodal evaluation after neoadjuvant therapy in patients with invasive lobular carcinoma (ILC), a diffusely growing tumor type. We retrospectively analyzed patients with stage 1–3 ILC who underwent pre-operative breast MRI after either neoadjuvant chemotherapy or endocrine therapy at our institution between 2006 and 2019. Two breast radiologists reviewed MRIs and evaluated axillary nodes for suspicious features. All patients underwent either sentinel node biopsy or axillary dissection. We evaluated sensitivity, specificity, negative and positive predictive values, and overall accuracy of the post-treatment breast MRI in predicting pathologic nodal status. Of 79 patients, 58.2% received neoadjuvant chemotherapy and 41.8% neoadjuvant endocrine therapy. The sensitivity and negative predictive value of MRI were significantly higher in the neoadjuvant endocrine therapy cohort than in the neoadjuvant chemotherapy cohort (66.7 vs. 37.9%, p = 0.012 and 70.6 vs. 40%, p = 0.007, respectively), while overall accuracy was similar. Upstaging from clinically node negative to pathologically node positive occurred in 28.0 and 41.7%, respectively. In clinically node positive patients, those with an abnormal post-treatment MRI had a significantly higher proportion of patients with ≥4 positive nodes on pathology compared to those with a normal MRI (61.1 versus 16.7%, p = 0.034). Overall, accuracy of breast MRI for predicting nodal status after neoadjuvant therapy in ILC was low in both chemotherapy and endocrine therapy cohorts. However, post-treatment breast MRI may help identify patients with a high burden of nodal disease (≥4 positive nodes), which could impact pre-operative systemic therapy decisions. Further studies are needed to assess other imaging modalities to evaluate for nodal disease following neoadjuvant therapy and to improve clinical staging in patients with ILC.