Mobility management is the key feature of cellular networks. When integrating unmanned aerial vehicles (UAVs) into cellular networks, their cell association needs to be carefully managed for coexistence with other cellular users. UAVs move in three dimensions and may traverse several cells on their flight path, and so may be subject to several handovers. In order to enable research on mobility management with UAV users, this paper describes the design, implementation, and testing methodology for handover experiments with aerial users. We leverage software-defined radios (SDRs) and implement a series of tools for preparing the experiment in the laboratory and for taking it outdoors for field testing. We use solely commercial offthe-shelf hardware, open-source software, and an experimental license to enable reproducible and scalable experiments. Our initial outdoor results with two SDR base stations connected to an open-source software core network, implementing the 4G long-term evolution protocol, and one low altitude UAV user equipment demonstrate the handover process.