The Methodological Quality Scale (MQS) for intervention programs: validity evidence

Abstract

IntroductionA wide variety of instruments are used when assessing the methodological quality (MQ) of intervention programs. Nevertheless, studies on their metric quality are often not available. In order to address this shortcoming, the methodological quality scale (MQS) is presented as a simple and useful tool with adequate reliability, validity evidence, and metric properties.MethodsTwo coders independently applied the MQS to a set of primary studies. The number of MQ facets was determined in parallel analyses before performing factor analyses. For each facet of validity obtained, mean and standard deviation are presented jointly with reliability and average discrimination. Additionally, the validity facet scores are interpreted based on Shadish, Cook, and Campbell’s validity model.Results and discussionAn empirical validation of the three facets of the MQ (external, internal, and construct validity) and the interpretation of the scores were obtained based on a theoretical framework. Unlike other existing scales, MQS is easy to apply and presents adequate metric properties. In addition, MQ profiles can be obtained in different areas of intervention using different methodologies and proves useful for both researchers doing meta-analysis and for evaluators and professionals designing a new intervention.

Methodological procedure based on quantitizing/liquefying: a case study to assess work climate in an emergency department

“…In the first QUAL stage, the observational design is proposed, and the indirect observation instrument is built, yielding a descriptive record that is systematized until it becomes a code matrix. In the QUAN stage, the systematized record is subjected to data quality control ( Chacón-Moscoso et al, 2023 ), using robust analytical techniques appropriate for categorical data. Finally, in the second QUAL stage, the results are interpreted.…”