Dietary fibers such as arabinoxylan (AX) are promising food constituents to prevent particular diet-related chronic diseases because of their prebiotic properties. Arabinoxylan fermentation by the gut microbiota depends on the structural architecture of AX, which can be modified during food processing and consequently affect its prebiotic potential, but it is little investigated. Therefore, the aim of this study was to evaluate the effects of naturally occurring and processing-induced structural alterations of the soluble AX of wheat bran and rye flour on the in vitro human colon fermentation. It was found that fermentation behavior is strongly linked to the AX fine structure and their processing-induced modifications. The short-chain fatty acid (SCFA) metabolism, acidification kinetics, bacterial growth, and bacterial composition revealed that wheat bran AX (WBAX) was fermented faster than rye flour AX. Increased levels of bound phenolic acids resulting from processing were identified as the inhibiting factor for AX fermentation kinetics. Bacterial genera promoted by AX varied between AX source and processing type, but also between microbiota. Extruded WBAX promoted butyrate production and growth of butyrate-producing Faecalibacterium in the butyrogenic microbiota while it did not enhance fermentation and inhibited the growth of Prevotella in the propiogenic microbiota. We anticipate that the findings of this study are a starting point for further investigation on the impact of processing-induced changes on the prebiotic potential of dietary fibers prior to human studies.