Microscopic polyangiitis (MPA) is an idiopathic autoimmune disease characterized by necrotizing vasculitis without granulomatous inflammation that predominantly affects small vessels. It most commonly presents in elderly patients but can occur at any age. Here we present a case of an elderly patient who presented with exertional dyspnea and fatigue in the setting of anemia and was diagnosed with microscopic polyangiitis. The patient had a past medical history significant for chronic kidney disease stage IIIb with worsening creatinine over the last four months along with feeling fatigued and intermittently dyspneic associated with black stools for the previous three weeks. A kidney biopsy revealed fibro-cellular crescents consistent with microscopic polyangiitis, and the patient was started on intravenous pulse dose steroids and later put on hemodialysis for worsening creatinine. He was later discharged home on rituximab and oral steroids. MPA is a rare vasculitis that presents with renal dysfunction and occasionally with pulmonary involvement. Gastrointestinal bleeding is rarely associated with the MPO and is thought to be caused by arterial aneurysms. Therefore, MPO should be considered in mind while evaluating GI bleeding in patients with worsening kidney function.