Purpose This study aims to investigate the formative evaluations of the training programmes of a Portuguese national railway public company for an entire calendar year. The aim is to uncover alternative configurations for the design of training programmes to create better levels of evaluation. This study is based on the following research question: What are the configurations that lead to the success and or failure of trainers and trainees? Among those, are there any common designs that generate the success and or failure of both trainers and trainees? Design/methodology/approach This study used matched data from an entire calendar year to examine the trainers and trainees’ evaluations of 429 training events. This study also used a fuzzy-set qualitative comparative analysis (fsQCA) to provide configurations that generate the success or failure of trainers and trainees. This methodology offers alternative pathways to the same outcomes and thus gives managers different options to reach similar results. Findings The results show that there are more configurations that lead to trainers’ success (five) than to its absence (four). However, the configurations that lead to trainees’ success (three) are less than those that lead to its absence (six). The findings indicate that a single common configuration exists that leads to high evaluations. Research limitations/implications This study does not address summative evaluations. Regarding data, the study acknowledges the use of self-evaluations for trainees, although they serve as a proxy for a learning evaluation. The generalisation of the results outside the Portuguese railway company’s context is not possible. Practical implications The proposed analysis is applicable to other settings without restrictions. Managers may replicate this study’s approach in their organisations to uncover the alternative configurations that lead to the success or failure of trainers and trainees. They may adopt the ones that lead to successful outcomes and avoid the ones that lead to undesired ones. Originality/value This study is innovative because it addresses concurrently the success or failure of trainers and trainees that is only possible by using the fsQCA method. This study opted to use this method to provide alternative pathways to extreme outcomes: the most successful or the most unsuccessful. These multiple pathways are better results compared to traditional quantitative statistical methods that only provide a single estimated solution to the presence of the dependent variable; for example, a regression analysis or structural equation modelling.

The evaluation of training is important because of the resources committed to training and the need for an effective performance by employees, but in practice, it is difficult to measure the return on training investment (ROI). Here, we follow the Kirkpatrick and Phillips model to measure the ROI from 327 training programs at the Portuguese national railway company in the calendar year 2016. Additionally, we use a fuzzy‐set Qualitative Comparative Analysis (fsQCA) on company records to determine the causal configurations that lead to higher values of ROI. In doing so, we add to the measurement literature but our major contribution is the use of fsQCA to provide a deeper analysis than hitherto of the factors associated with higher or lower ROI. The findings show the importance of complex causation for ROI and offer five alternative configurations of conditions that lead to higher ROI values. There are only three configurations of conditions that lead to lower ROI values. Further, the results show the importance of organizing training programs in small groups. Although the results cannot be generalized due to the use of national data in the study, the proposed method of analysis is generally applicable.