POI(point of interest) recommendation is a very necessary research field in both academic and commerce, however, predicting users’ potential points of interest is always faced with the problems of data sparsity and context semantics. Some studies have shown that graph embedding technology alleviates the problem of data sparsity to a certain extent. However, neither graph embedding techniques nor unsupervised learning models can adaptively learn the different effects of multiple relations between users and POIs, respectively. In view of this, we leverage the contextual information of users and POIs to build the multi-view affinity graphs(e.g. User-User, POI-POI and User-POI), and learn the latent representations of users and POIs based on the Graph Embedding technology and Attention mechanism, namely the GEA model. In particular, we first construct multi-view affinity graphs by using user’s social relationship, geographical distance and check-in behaviour, and embed them into a low dimensional shared space to learn the latent representation of users and POIs. Afterwards, in order to take advantage of the different effects of multiple relationships in the final recommendation task, we exploit the attention mechanism to obtain the fused latent representation and make recommendation according users’ potential preferences. Finally, we design a multi-task objective function for joint optimization to obtain more accurate recommendation results. Extensive experiments on Gowalla have verified the effectiveness of our model.