In recommender systems, measuring user similarity is essential for predicting a user's ratings on items. Most traditional works calculate the similarity based on historical ratings shared between two users, without considering the probability of users' different behaviors. To address this issue, our work designs a similarity measure based on the users' historical behavior probabilities. Based on the similarity method, a complex network of user relationships is modeled. The user degree and community information of the modeled network, as well as the number of shared ratings between users, are used with the proposed similarity measure to design a rating prediction algorithm for recommender systems. Experiments based on MovieLens and Netflix data sets show that this method is effective and can successfully improve the accuracy of rating predictions and reduce the number of neighbors required to achieve the optimal prediction accuracy. Our research shows that in a complex system, the relationship between users can be effectively measured by the users' historical behavior probability, providing a new perspective for future research on similarity measurement.INDEX TERMS complex network modeling, recommender systems, degree centrality, link prediction, user behavior probability.