Psychometric assessment of an instrument evaluating the effects of affective variables on students’ WTC in face-to-face and digital environment

Abstract

The current study was aimed to translate and examine the psychometric characteristics of the Indonesian version of Lee and Hsieh's (2019) questionnaire measuring the effects of four affective variables (i.e. self-confidence, L2 anxiety, grit, and motivation) on students' willingness to communicate in face-to-face (F2F) in both inside and outside classrooms, and in a digital environment. Data were collected from 458 students-269 university students, 102 upper secondary school students, and 87 lower secondary school students. Several statistical analyses (i.e. unidimensionality assessment, reliability analysis, misfit analysis, and differential item functioninganalysis) was performed using Rasch analysis. Findings of the study revealed that seven of eight constructs in the Indonesian version of Lee and Hsieh's (2019) questionnaire is valid with Indonesian samples, and they preserve the psychometric characteristics of the original scale. Recommendations are offered in reference to the findings.

A Rasch-based validation of EFL teachers’ received online social support scale

“…In our analysis, the respondent logit data with z-standardised (Zstd) infit and outfit mean square less than −2 or above +2 were observed to misfit. Of 214 samples, 114 were shown to misfit with the Rasch model and thus were excluded for the second round analysis (Huang et al, 2020;Linacre, 2002bLinacre, , 2010Mulyono et al, 2020). It is worth noting that the amount of 100 samples as used in the current study still met the minimum requirements for sample size in Rasch analysis, which is 50 samples (Linacre, 1994).…”

“…To this end, differential item functioning (DIF) was evaluated using Rasch-Welch tests. DIF occurs when the DIF contrast value is more significant than 0.5 logits and is observed to be significant (Rasch-Welch p < .05) (Chan & Subramaniam, 2020;Linacre, 2018;Ling Lee et al, 2020;Mulyono et al, 2020). In the current study, DIF was examined in reference to the participants' demographic aspects, including gender, age, background of education, teaching experiences, and level of teaching.…”

“…Then, the second author, who was fluent in English and Bahasa Indonesia, validated the translated questionnaire and proofread it. As suggested in the literature, the wordings of the Indonesian translated questionnaire were evaluated and refined to improve the questionnaire readability (Mulyono et al, 2020).…”

“…A Rasch-based analysis was adopted to provide valid evidence for the WSSC. Rasch analysis has been widely used for instrument validations and maintains the instrument quality (Mulyono et al, 2020;Yu, 2020). Rasch model analysis is viewed as an analysis of a latent trait model developed to reflect "the probability that a person provides a certain response to an item as a function of the person and item characteristics" (Colledani et al, 2020, p. 2).…”

The application of WhatsApp to support online learning during the COVID-19 pandemic in Indonesia

“…It is important to note that the current study does not aim to examine the relationships between TAM's external and internal variables as in the previous studies. Instead, using Rasch model analysis, the current study aimed to determine the strength of students' responses ( Goh et al., 2010 ; Mulyono et al., 2020 ; Ningsih et al., 2021 ). More importantly, Rasch model analysis allows the researcher to map and detail the students' responses to the TAM questionnaire's corresponding items.…”