(1) Background: Mobile phone use during gait is associated with adverse health outcomes, namely increased risk of pedestrian injury. Healthy individuals can voluntarily prioritize concurrent task performance, but the factors underlying the impact of phone use during walking remain largely unknown. Thus, the objective of this work was to evaluate the relationship between subjective (perceived) prioritization, cognitive flexibility and dual-task performance when using a mobile phone during walking. (2) Methods: Thirty young participants walked for one minute with and without reading or texting on a mobile phone, as well as reading or texting while sitting. Walking performance (kinematics) was recorded, as well as phone use (text comprehension, text read/written), mental workload, perceived prioritization (visual analog scale), and cognitive flexibility (trail-making test). (3) Results: Texting while walking was associated with larger decreases in gait speed, larger gait variability, higher mental workload, and lower text comprehension compared to reading. Perceived prioritization was associated with walking dual-task costs (DTCs) (r = 0.39–0.42, p < 0.04) when texting, and better cognitive flexibility was associated with lower gait DTCs when texting (r = 0.55, p = 0.002) but not reading. (4) Conclusions: The context-dependent link between perceived prioritization, cognitive flexibility, and walking DTCs promotes our understanding of the factors underlying texting-while-walking performance. This could identify individuals who are more prone to dual-task interference in this increasingly common and dangerous task.