In general, multilayer networks are often a significantly more apt description of reallife systems than isolated or single networks. In this paper, we explore the effect of memory on the evolution of prisoner's dilemma (PD) game by constructing different kinds of two-layer networks. The results show that the heterogeneous network structure is conducive to promoting individuals to adopt cooperative behaviors. However, as the lure income T increases, the individuals who take cooperative behavior in the entire system gradually decrease. Further research shows that if no more than one layer of network presents large heterogeneity, then the less the individuals are affected by historical gains, the better the cooperation will be among individuals. By contrast, if both layers of networks are less heterogeneous, the greater the impact of historical returns on individuals, the easier it is for cooperation between individuals. Furthermore, if individuals change their strategies mainly by imitating the strategies of their neighbors, then it is beneficial to promote cooperation among individuals in the entire system. However, it is not conducive to cooperation among individuals if individuals change their own strategy mainly through strategies of their counterparts. The final result indicates that, if at least one layer is a heterogeneous network structure, the cooperation between individuals in the entire system will be blocked when the length of the individual's historical memory is too long.