The popularity of big data from cell phone communication records makes cellular handoff probe system become an optimal choice for the collection of traffic speeds, especially for the large-scale freeway network. As a probe-based technique, the issues related to sample size are vital for the accuracy of cellular handoff probe system. Based on the current findings from simulation and field test, this study made an extensive discussion about the sample size problems using the field data. The subsequent investigation began with the correlation analysis between the sample size and the potentially influencing factors (the length of handoff link and traffic volume). Furthermore, a Weibull distribution model was set up to fit sample size for different handoff links. And then, the impact of sample size on the accuracy of cellular handoff probe system was following. Finally, based on the above explorations, several concluding remarks were presented as a reference for determining appropriate sample size to achieve high precision.