Objective Mis-triage may have serious consequences for patients in mass casualty incidents (MCI) at sea. The purpose of this study was to assess outcome, reliability and validity of an analogue and a digital recording system for triage of a MCI at sea. Methods The study based on a triage exercise conducted with a cross-over-design. Forty-eight volunteers were presented a fictional MCI with 50 cases. The volunteers were randomly assigned to start with the analogue (Group A, starting with the analogue followed by the digital system) or digital system (Group B, starting with the digital followed by the analogue system). Triage score distribution and agreement between the triage methods and a predefined standard were reported. Reliability was analysed using Cronbach's Alpha and Cohen's Kappa. Validity was measured through sensitivity, specificity and predictive value. Treatment, period and carry-over-effects were analysed using a linear mixed-effects model. Results The number of patients triaged (total: n = 3545) with the analogue system (n = 1914; 79.75%) was significantly higher (p = 0.001) than with the digital system (n = 1631; 67.96%). A trend towards a higher percentage of correct triages with the digital system was observed (p = 0.282). Ratio of under-triage was significantly smaller with the digital system (p = 0.001). Validity measured with Cronbach's Alpha and Cohen's Kappa was higher with the digital system.