BackgroundFailure mode and effects analysis (FMEA) is a valuable tool for radiotherapy risk assessment, yet its outputs might be unreliable due to failures not being identified or due to a lack of accurate error rates.PurposeA novel incident reporting system (IRS) linked to an FMEA database was tested and evaluated. The study investigated whether the system was suitable for validating a previously performed analysis and whether it could provide accurate error rates to support the expert occurrence ratings of previously identified failure modes.MethodsTwenty‐three pre‐identified failure modes of our external beam radiotherapy process, covering the process steps from patient admission to treatment delivery, were proffered on dedicated FMEA feedback and incident reporting terminals generated by the IRS. The clinical setting involved a computed tomography scanner, dosimetry, and five linacs. Incoming reports were used as basis to identify additional failure modes or confirm initial ones. The Kruskal–Wallis H test was applied to compare the risk priorities of the retrospective and prospective failure modes. Wald's sequential probability ratio test was used to investigate the correctness of the experts’ occurrence ratings by means of the number of incoming reports.ResultsOver a 15‐month period, 304 reports were submitted. There were 0.005 (confidence interval [CI], 0.0014–0.0082) reported incidents per imaging study and 0.0006 (CI, 0.0003–0.0009) reported incidents per treatment fraction. Sixteen additional failure modes could be identified, and their risk priorities did not differ from those of the initial failure modes (p = 0.954). One failure mode occurrence rating could be increased, whereas the other 22 occurrence ratings could not be disproved.ConclusionsOur approach is suitable for validating FMEAs and deducing additional failure modes on a continual basis. Accurate error rates can only be provided if a sufficient number of reports is available.
