Executive function and processing speed difficulties are observed in children living with sickle cell anemia (SCA). The influence of processing speed on executive function is not well understood. We recruited 59 children living with SCA and 24 matched controls aged 8–18 years between 2010 and 2016 from clinics in the UK. Children completed tests in processing speed and cognitive flexibility, subdomains of executive function. MRI scans were conducted within one year of testing; oxygen saturation was obtained on the day of testing. Hemoglobin levels were obtained from medical records. Caregivers completed the executive function questionnaire. Hierarchical linear regressions found that hemoglobin, oxygen saturation, age, infarct status, and processing speed were not independent predictors for any model. However, for all cognitive flexibility tests, there was a significant interaction between infarct status and processing speed; children without silent cerebral infarction (SCI) with faster processing speed had better cognitive flexibility. Our findings indicate that, when interpreting executive function difficulties, it is important to account for the relationship between SCI status and processing speed. More research is needed to elucidate the mechanisms, but clinically, including executive function testing as part of clinic visits by embedding psychologists within the healthcare team would appear to be a critical step.