Concurrent Cerebral Amyloid Angiopathy (CAA) and Atrial Fibrillation are becoming an increasingly common dilemma in clinical practice due to the aging population and the comorbidities associated with it. In such patients, the physician must appreciate and strike the difficult balance between the risk of ischemic strokes from atrial fibrillation on one hand, and that of intracerebral hemorrhage from coexisting CAA on the other. Anticoagulation is necessary for the former but potentially deleterious for the latter. In this case report, we present the case of a 67-year-old woman with a long history of atrial fibrillation on rivaroxaban who recently began to experience recurrent transient neurological deficits that were later diagnosed as amyloid spells related to concomitant CAA. While there is no clear-cut consensus in published literature on how to best manage these patients regarding the use of anticoagulation, it is recommended to involve a multidisciplinary team for optimal management of these patients.