Background Adequate training and preparation of medical first responders (MFRs) are essential for an optimal performance in highly demanding situations like disasters (e.g., mass accidents, natural catastrophes). The training needs to be as effective as possible, because precise and effective behavior of MFRs under stress is central for ensuring patients’ survival and recovery. This systematic review offers an overview of scientifically evaluated training methods used to prepare MFRs for disasters. It identifies different effectiveness indicators and provides an additional analysis of how and to what extent the innovative training technologies virtual (VR) and mixed reality (MR) are included in disaster training research. Methods The systematic review was conducted according to the PRISMA guidelines and focused specifically on (quasi-)experimental studies published between January 2010 and September 2021. The literature search was conducted via Web of Science and PubMed and led to the inclusion of 55 articles. Results The search identified several types of training, including traditional (e.g., lectures, real-life scenario training) and technology-based training (e.g., computer-based learning, educational videos). Most trainings consisted of more than one method. The effectiveness of the trainings was mainly assessed through pre-post comparisons of knowledge tests or self-reported measures although some studies also used behavioral performance measures (e.g., triage accuracy). While all methods demonstrated effectiveness, the literature indicates that technology-based methods often lead to similar or greater training outcomes than traditional trainings. Currently, few studies systematically evaluated immersive VR and MR training. Conclusion To determine the success of a training, proper and scientifically sound evaluation is necessary. Of the effectiveness indicators found, performance assessments in simulated scenarios are closest to the target behavior during real disasters. For valid yet inexpensive evaluations, objectively assessible performance measures, such as accuracy, time, and order of actions could be used. However, performance assessments have not been applied often. Furthermore, we found that technology-based training methods represent a promising approach to train many MFRs repeatedly and efficiently. These technologies offer great potential to supplement or partially replace traditional training. Further research is needed on those methods that have been underrepresented, especially serious gaming, immersive VR, and MR.