With the ever-growing demand for even higher throughput, ultradense networks (UDNs) are being deployed for the fifth generation (5G) mobile communications. Although massively distributed radio access points (APs) result in a considerable increase in throughput, they also cause some critical problems. When employing a wireless backhaul, the backhaul capacity becomes a limiting factor, which may result in a high packet loss rate. Furthermore, dense deployment of APs leads to more frequent handoffs for mobile user equipment, which results in heavy measurement and signaling overhead. For a better trade-off between the packet loss rate and the handoff overhead, a machine learning approach for flexible virtual cell (VC) design is proposed that leverages particle swarm optimization (PSO) to quickly find the optimal VC solution. To be responsive to the dynamic traffic demand and backhaul capacity of APs, a new parameter called "weighted distance" is employed in the modified K-means algorithm, which is nested in the PSO procedures for master AP selection and VC boundary determination. Compared with an exhaustive search, optimal VC solutions can be found efficiently through considerably fewer iterations. The proposed method is generic and applicable to disparate UDN application scenarios.INDEX TERMS K-means clustering, mobility management, particle swarm optimization, resource management, ultradense networks, virtual cell.