We investigate the flow of unsteadfy three-dimensional viscoelastic fluid through an array of symmetric and asymmetric sets of cylinders constituting a model porous medium. The simulations are performed using a finite-volume methodology with a staggered grid. The solid-fluid interfaces of the porous structure are modeled using a second-order immersed boundary method [S. De et al., J. Non-Newtonian Fluid Mech. 232, 67 (2016)]. A finitely extensible nonlinear elastic constitutive model with Peterlin closure is used to model the viscoelastic part. By means of periodic boundary conditions, we model the flow behavior for a Newtonian as well as a viscoelastic fluid through successive contractions and expansions. We observe the presence of counterrotating vortices in the dead ends of our geometry. The simulations provide detailed insight into how flow structure, viscoelastic stresses, and viscoelastic work change with increasing Deborah number De. We observe completely different flow structures and different distributions of the viscoelastic work at high De in the symmetric and asymmetric configurations, even though they have the exact same porosity. Moreover, we find that even for the symmetric contraction-expansion flow, most energy dissipation is occurring in shear-dominated regions of the flow domain, not in extensional-flow-dominated regions.