Miniaturization has made ultrasound (US) technology ultraportable and, in association with their relatively low cost, made handheld devices attractive for medical education training programs. However, performing an ultrasound requires complex skills, and it is unclear whether handheld devices are suitable for the training of novices. Our study aimed to identify to what extent handheld US devices can be employed in medical undergraduates’ and residents’ education. We selected studies that evaluate the results obtained by students and/or residents who have participated in ultrasound training programs using handheld devices. The studies were included if they reported post-test (pre-test optional) achievements or a comparison with a control group (a group of experts or novices who underwent a different intervention). Twenty-six studies were selected, and their characteristics were summarized. Handheld ultrasound devices were used in training programs to learn echocardiography, abdominal, and/or musculoskeletal ultrasound. Statistically significant progress was noted in the ability of naïve participants to capture and interpret ultrasound images, but training duration influenced the outcomes. While ultrasound training using handheld devices has proven to be feasible for various body regions and purposes (e.g., better understanding of anatomy, clinical applications, etc.), the long-term impacts of handheld education interventions must be considered in addition to the short-term results to outline guidelines for targeted educational needs.