The physical mechanism underlying the mechanical behavior of glassy polymers has been studied over decades but remains a long‐standing issue. A consensus view achieved is that the yield, flow, and stress relaxation behaviors are due to structural relaxation in the polymer mainly caused by chain conformation transitions. This is the key physical idea behind the many existing elastic–plastic constitutive models for glassy polymers. In this paper, such a constitutive model was employed for predicting and analyzing the stress relaxation of a glassy polymer. It is found that the model works well in predicting the pre‐yield stress relaxation but significantly underestimates the post‐yield stress relaxation. As considering the chain conformation transition alone leads to a dilemma for the model to concurrently represent the yield/flow and stress relaxation behaviors, the model was extended to incorporate an additional structural relaxation mechanism assumed to originate from the dissociation of weak linkages in the chain network. The extended model succeeds in concurrently representing the yield/flow, and stress relaxation behaviors in the whole deformation region, of which the reasons were analyzed. The knowledge revealed in this paper is instructive and may shed new light on understanding the structural relaxation and mechanical behavior of glassy polymers.