The goal of the session-based recommendation system (SBRS) is to predict the user’s next behavior based on anonymous sessions. Since long-term historical information of users is not available, deep learning technology has become the mainstream technology in session-based recommendation systems instead of traditional content-based recommendation methods. However, most SBRS methods only consider the session itself, ignoring the collaborative information from other sessions. Even if some SBRS models consider collaborations between sessions, they mostly use the click order to calculate the similarity only and ignore the time the user spends on different items, which might imply the user’s varying interest on these items. In this paper, we propose a session-based recommendation model with GNN and time-aware memory networks (SR-GTM), which learns the user’s interest representation by combining the information from the session itself and the collaborative information from relevant neighbor sessions. Specifically, SR-GTM mainly includes inner feature extraction module (IFEM) and outer feature extraction module (OFEM). IFEM uses GNN to learn the session features based on its item sequence, and OFEM uses a memory network with dwell time information encoded to extract collaborative information. Finally, SR-GTM aggregates IFEM and OFEM by the gating mechanism and then decodes the output by a softmax layer to obtain the recommendation score for each candidate item. Experiments on three public datasets Yoochoose1/64, Yoochoose1/4, and RetailRocket show that SR-GTM achieves optimal performance compared with other state-of-the-art methods. More specifically, SR-GTM has improvements of 0.77%, 0.38%, and 3.63% over the best baseline method in P@20 and has improvements of 2.91%, 2.52%, and 2.49% in MRR@20, respectively.