Existing IR-based expert finding generally follows two methods, i.e. the profile-based method and the voting-based one. However, neither the expert-relevant data collected in the profilebased method nor the query-relevant data used for the votingbased method is completely accurate within the confines of current relevance ranking approaches. This problem has been rarely discussed, but impedes expert finding. On this issue, we provide a feasible solution, that is, the collection can be filtered to generate a subset of high-precision relevant data for further processing. In this paper, we propose two perspectives of filtering approaches, i.e. the query-centered perspective and the expert-centered one. For both perspectives, some specific strategies are also discussed and experimented under the CERC collection using the TMJAC model, a voting-based method. On such basis, the different preferences of two perspectives are revealed. Further, to examine the stability of filtering, we examine the filtering strategies using a profile-based method and also testify the effects under the W3C collection. In conclusion, the filtering we proposed is a universal approach of improving expert finding performance.