Web service recommendation is of great importance when users face a large number of functionally-equivalent candidate services. To recommend Web services that best fit a user's need, QoS values which characterize the non-functional properties of those candidate services are in demand. But in reality, the QoS information of Web service is not easy to obtain, because only limited historical invocation records exist. To tackle this challenge, in recent literature, a number of QoS prediction methods are proposed, but they still demonstrate disadvantages on prediction accuracy. In this paper, we design a location-based hierarchical matrix factorization (HMF) method to perform personalized QoS prediction, whereby effective service recommendation can be made. We cluster users and services into several user-service groups based on their location information, each of which contains a small set of users and services. To better characterize the QoS data, our HMF model is trained in a hierarchical way by using the global QoS matrix as well as several location-based local QoS matrices generated from userservice clusters. Then the missing QoS values can be predicted by compactly combining the results from local matrix factorization and global matrix factorization. Comprehensive experiments are conducted on a real-world Web service QoS dataset with 1,974,675 real Web service invocation records. The experimental results show that our HMF method achieves higher prediction accuracy than the state-of-the-art methods.