Student’s Academic Efficacy or Inefficacy? An Example on How to Evaluate the Psychometric Properties of a Measuring Instrument and Evaluate the Effects of Item Wording

Abstract

This study evaluated the effect of item inversion on the construct validity and reliability of psychometric scales and proposed a theoretical framework for the evaluation of the psychometric properties of data gathered with psychometric instruments. To this propose, we used the Maslach Burnout Inventory, which is the most used psychometric inventory to measure burnout in different professional context (Students, Teachers, Police, Doctors, Nurses, etc…). The version of the MBI used was the MBI-Student Survey (MBI-SS). This inventory is composed of three key dimensions: Exhaustion, Cynicism and Professional Efficacy. The two first dimensions-which have positive formulated items-are moderate to strong positive correlated, and show moderate to strong negative correlations with the 3rd dimension-which has negative formulated items. We tested the hypothesis that, in college students, formulating the 3rd dimension of burnout as Inefficacy (reverting the negatively worded items in the Efficacy dimension) improves the correlation of the 3rd dimension with the other two dimensions, improves its internal consistency, and the overall MBI-SS' construct validity and reliability. Confirmatory factor analysis results, estimated by Maximum Likelihood, revealed adequate factorial fit for both forms of the MBI-SS (with Efficacy) vs. the MBI-SSi (with Inefficacy). Also both forms showed adequate convergent and discriminant related validity. However, reliability and convergent validity were higher for the MBI-SSi. There

University Student Engagement Inventory (USEI): Transcultural Validity Evidence Across Four Continents

“…Item 6 has previously been identified as a problematic item in this scale as it is the only reverse-coded item (Sinval et al, 2018). It can be positively worded as "I feel very accomplished at this school" because reversed items may have reduced sensitivity (as demonstrated by Maroco et al, 2014) for the efficacy dimension of the MBI-SS. Item 11 ("When I read a book, I question myself to make sure I understand the subject I'm reading about") refers to reading a book and may not have the same relevance across different courses or education systems as many students may use a diverse set of media and often read specific chapters of books.…”

“…In its original formulation (Schaufeli et al, 2002), the Efficacy dimension has its items positively worded while Emotional Exhaustion and Cynicism are composed of negatively worded items. Here we use a version of the MBI-SS (MBI-SSi; Maroco et al, 2014) where the items in the Efficacy dimension were negatively worded to give rise to the Inefficacy (INEF) dimension. Four versions of the scale were used in this study: Portuguese (Portugal, Brazil, Mozambique), English (the United Kingdom, United States, and Finland), Serbian (Serbia), and simplified Chinese (Macau and Taiwan).…”

Predictors of academic efficacy and dropout intention in university students: Can engagement suppress burnout?

“…A minimum sample size of 200 was shown to be enough to attain bias below 1% for both point estimates and standard errors of the parameters; 99% confidence interval coverage greater than 95%, and minimum power of 90%. However, to ensure that the study sample, which was non–probabilistic, would capture a large amount of the normative population variance we set the sample size at a minimum of 300 students per country/region corresponding to 20 participants per item of the full SEM model as suggested by Marôco [ 51 ]. A non-probabilistic convenience sampling method was chosen to achieve this sample size.…”

Análisis complementario del MBI-SS en el contexto académico colombiano

“…Cabe precisar que es esperable algún grado de complejidad factorial cuando son evaluados instrumentos multidimensionales, por lo que los indicadores presentados, hasta cierto punto, podrían resultar predecibles. Con todo, hallazgos como estos deben ser reportados y discutidos debido a que en algunos estudios previos la correlación interfactorial observada entre Agotamiento y Cinismo es la más elevada (Hu & Schaufeli, 2009;Maroco, Maroco, & Campos, 2014;Schaufeli, Martinez, Pinto, Salanova, & Bakker, 2002), llegando inclusive a la multicolinealidad (p.e., Tsubakita & Shimazaki, 2016).…”