Viscoelastic flows through porous media become unstable and chaotic beyond critical flow conditions, impacting widespread industrial and biological processes such as enhanced oil recovery and drug delivery. Understanding the influence of the pore structure or geometry on the onset of flow instability can lead to fundamental insights into these processes and, potentially, to their optimization. Recently, for viscoelastic flows through porous media modeled by arrays of microscopic posts, Walkama et al. [D. M. Walkama, N. Waisbord, J. S. Guasto, Phys. Rev. Lett. 124, 164501 (2020)] demonstrated that geometric disorder greatly suppressed the strength of the chaotic fluctuations that arose as the flow rate was increased. However, in that work, disorder was only applied to one originally ordered configuration of posts. Here, we demonstrate experimentally that, given a slightly modified ordered array of posts, introducing disorder can also promote chaotic fluctuations. We provide a unifying explanation for these contrasting results by considering the effect of disorder on the occurrence of stagnation points exposed to the flow field, which depends on the nature of the originally ordered post array. This work provides a general understanding of how pore geometry affects the stability of viscoelastic porous media flows.