In numerical computation of ladle metallurgy, multiphase models are essential. Still, these models are afflicted with great uncertainty, making a validation with experimental data mandatory. Validation experiments fundamentally differ from physical modeling experiments because emphasis is on a complete documentation of all boundary conditions and a detailed uncertainty assessment. For this work, the experimental design for a comprehensive accuracy assessment of numerical models and reported quantities were jointly determined with international numerical experts. The framework comprises a plume analysis and flow measurements in the single- and multiphase regions of a water model with defined conditions. All influencing factors are documented extensively and their contribution to the overall data uncertainty is quantified. Detailed data are made publicly available within a validation benchmark database for isothermal multiphase flow in metallurgical ladles. A first draft of a standardized validation procedure, including a single number validation score, is proposed. Using the available data, the accuracy of numerical models can be quantified more accurately and comparably, which helps in advancing the model’s further development. It also lays the foundation for a standardization of the validation process, which can lead to greater acceptance of the numerical results.