In this paper, we propose a machine learning−based mobility robustness optimization algorithm to optimize handover parameters for seamless mobility under dynamic small-cell networks. Small cells can be arbitrarily deployed, portable, and turned on and off to fulfill wireless traffic demands or energy efficiency. As a result, the small-cell network topology dynamically varies challenging network optimization, especially handover optimization. Previous studies have only considered dynamics due to user mobility in a specific static network topology. To optimize handovers under dynamic network topologies, together with user mobility, we propose an algorithm consisting of two steps: topology adaptation and mobility adaptation. To adapt to a dynamic topology, the algorithm obtains prior knowledge, which presents a belief distribution of the optimal handover parameters, for the current network topology as coarse optimization. In the second step, the algorithm fine-tunes the handover parameters to adapt to user mobility based on reinforcement learning, which utilizes the knowledge obtained during the first step. Under a dynamic small-cell network, we showed that the proposed algorithm reduced adaptation time to 4.17% of the time needed by a comparative machine-based algorithm. Furthermore, the proposed algorithm improved the user satisfaction rate to 416.7% compared to the previous work.INDEX TERMS Transfer learning, distributed reinforcement learning, small cell on/off, self-organizing network, handover optimization.