In seismic data processing, accurate and efficient automatic velocity picking algorithms can significantly accelerate the processing, and the main branch is to use velocity spectra for velocity pickup. Recently, machine learning algorithms have been widely used in automatic spectrum picking. Even though deep learning methods can address the problem well in supervised cases, they are often accompanied by expensive computational costs and low interpretability. On the contrast, unsupervised learning methods based on the physical knowledge have great potential to efficiently resolve the task. In this paper, we propose an unsupervised ensemble learning (UEL) method to pick the root mean square (RMS) velocities on the spectrum. In particular, UEL utilizes the information of nearby velocity spectra and the nearest seed velocity curve to assist the selection of effective and reasonable velocity points. To increase the coherence of energy peaks, an information gain method is developed by local normalization. In addition, we designed the attention scale-space filter (ASSF) clustering method to incorporate the coherence information into the picking process. Experiments on three datasets demonstrate that compared to traditional clustering methods, UEL can recognize energy clusters better, especially with smaller blobs. Moreover, the injection of nearby spectra and interval velocity constraint in UEL significantly improves the robustness and accuracy of picking results.