For the first time, this research addresses the assessment of the quality of knowledge embedded in beach quality indexes from a socioecological perspective. We took the most widespread beach quality indexes and identified, selected and assessed the most important existing assumptions. We scored the robustness of these assumptions, using an inclusive methodology (stakeholder meeting, four focus groups and an online questionnaire). The NUSAP criteria for assessing the value-ladenness of scientific studies (Influence of resource limitations, (Im)Plausibility, Choice space, Agreement among peers, Analysts' subjectivity and Influence on global results) were contrasted and discussed. A final list of the 10 weakest assumptions was presented and discussed. Most of these assumptions are fairly robust, but attention should mainly focus on their influence on global outcomes and (im)plausibility, as the weakest scored criteria. The choice space scores revealed the possibility of including new alternatives to the assumptions, when necessary. Assumptions loaded with framing concepts are weaker than those linked to more concrete objectives. We detected dissociation between the discourse and the operational development of the indexes, in which the narratives prioritizing user satisfaction are predominant and scientific data analysis is often decontextualized. We therefore suggest that science should be opened up throughout the building process of indexes: from the identification of problems to the reporting of results and related uncertainties. The NUSAP method proved to be useful for identifying weak points in beach quality indexes.