Quality of Service (QoS) prediction for Web services is a hot research topic in the field of services computing. Recently, representation learning of heterogeneous networks has attracted much attention, and specifically the relationship between users and services, as a typical heterogeneous network in which heterogeneity and rich semantic information provide a new perspective for QoS prediction. This paper proposes a novel QoS Prediction scheme based on a heterogeneous graph attention network. Our method first unitizes the user's location information to construct an attributed user-service network. Then, considering the difference between nodes and links in the latter network, we model a heterogeneous graph neural network based on a hierarchical attention machine (HGN2HIA) that includes node-and semanticlevel attentions. Specifically, node-level attention aims to learn the importance between a node and its meta-path-based neighbors, while semantic-level attention learns the importance of different meta-paths. Finally, user embedding will be generated by aggregating features from meta-path-based neighbors in a hierarchical manner, used for QoS prediction. Experimental results on the public WS-Dream dataset demonstrate the superior performance of the proposed model over the current state-of-the-art methods, with NMAE and RMSE metrics reduced by at least 2.56% and 1.3%, respectively. Furthermore, the experimental results highlight that node-level attention contributes more than semantic-level. Overall, we demonstrate that introducing these attention levels improves the QoS prediction performance.