Recommender systems are important approaches for dealing with the information overload problem in big data era, and various kinds of auxiliary information, including time and sequential information, can help to improve the performance of retrieval and recommendation tasks. However, it is still a challenging problem how to fully exploit such information to achieve high-quality recommendation results and improve users' experience. In this work, we present a novel sequential recommendation model named Multivariate Hawkes Process Embedding with attention (MHPE-a), which combines a temporal point process with attention mechanism to predict the items that the target user may interact with according to her/his historical records. Specifically, the proposed approach MHPE-a can model users' sequential patterns in their temporal interaction sequences accurately with a multivariate Hawkes process. Then, we perform accurate sequential recommendation to satisfy target users' realtime requirement based on their preferences obtained with MHPE-a from their historical records. Especially, an attention mechanism is used to leverage users' long/short-term preferences adaptively to achieve accurate sequential recommendation. Extensive experiments are conducted on two real-world datasets (lastfm and gowalla), and the results show that MHPE-a achieves better performance than state-of-the-art baselines.