Broca’s area in the posterior half of the left inferior frontal gyrus has long been thought to be critical for speech production. The current view is that long-term speech production outcome in patients with Broca’s area damage is best explained by the combination of damage to Broca’s area and neighbouring regions including the underlying white matter, which was also damaged in Paul Broca’s two historic cases. Here, we dissociate the effect of damage to Broca’s area from the effect of damage to surrounding areas by studying long-term speech production outcome in 134 stroke survivors with relatively circumscribed left frontal lobe lesions that spared posterior speech production areas in lateral inferior parietal and superior temporal association cortices. Collectively, these patients had varying degrees of damage to one or more of nine atlas-based grey or white matter regions: Brodmann areas 44 and 45 (together known as Broca’s area), ventral premotor cortex, primary motor cortex, insula, putamen, the anterior segment of the arcuate fasciculus, uncinate fasciculus and frontal aslant tract. Spoken picture description scores from the Comprehensive Aphasia Test were used as the outcome measure. Multiple regression analyses allowed us to tease apart the contribution of other variables influencing speech production abilities such as total lesion volume and time post-stroke. We found that, in our sample of patients with left frontal damage, long-term speech production impairments (lasting beyond 3 months post-stroke) were solely predicted by the degree of damage to white matter, directly above the insula, in the vicinity of the anterior part of the arcuate fasciculus, with no contribution from the degree of damage to Broca’s area (as confirmed with Bayesian statistics). The effect of white matter damage cannot be explained by a disconnection of Broca’s area, because speech production scores were worse after damage to the anterior arcuate fasciculus with relative sparing of Broca’s area than after damage to Broca’s area with relative sparing of the anterior arcuate fasciculus. Our findings provide evidence for three novel conclusions: (i) Broca’s area damage does not contribute to long-term speech production outcome after left frontal lobe strokes; (ii) persistent speech production impairments after damage to the anterior arcuate fasciculus cannot be explained by a disconnection of Broca’s area; and (iii) the prior association between persistent speech production impairments and Broca’s area damage can be explained by co-occurring white matter damage, above the insula, in the vicinity of the anterior part of the arcuate fasciculus.