This paper proposes a data-driven method of mmWave beam selection in multi-cell systems to achieve a near-optimal fast beam allocation with low complexity. In particular, an online learning algorithm based on support vector machine (SVM) equipped with the radial basis function kernel, namely SVM-based online beam selection (SBOS) algorithm is proposed. The proposed algorithm starts with an adaptive beam selection process for certain traffic pattern that uses an SVM learning model to adaptively refine the beam selection strategy. Specifically, SVM-based model labels the feedback (the average information rate) from the cellular system, then learns from samples, and makes the scheme space smaller by maximising samples' minimum distances to all labelled samples in the sample space constrained by newly learned boundaries. Then, according to the aggregated data about the traffic patterns and the performance of corresponding beam selection strategy, SBOS algorithm exploits beam selection schemes recorded in the database or explores new schemes for unknown situations, respectively, and how to tune the hyperparameters for the SBOS algorithm is discussed. Furthermore, the extensive simulation results show that the proposed algorithm achieves a better performance versus upper confidence bound and Random methods. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.