SummaryWhen selecting services from a cloud center to build applications, the quality of service (QoS) is an important nonfunctional attribute to be considered. However, in actual application scenarios, the QoS details for many services may not be available. This has led to a situation where prediction of the missing QoS records for services has become a key problem for service selection. This article presents a selective ensemble learning (SEL) framework for prefilling‐based matrix factorization (PFMF) predictors. In each PFMF predictor, the improved collaborative filtering is defined by examining the stability of the QoS records when measuring the similarity of users (or services), and then used to prefill empty records in the initial QoS matrix. To ensure the diversity of the basic PFMF predictors, various prefilled QoS matrices are constructed for the matrix factorization. In this process, different reference weights are assigned to the original and the prefilled QoS records. Finally, particle swarm optimization is used to set the ensemble weights for the basic PFMF predictors. The proposed SEL on PFMF (SEL‐PFMF) algorithm is validated on a public dataset, where its prediction performance outperforms the state‐of‐the‐art algorithms, and also shows good stability.