Design and Evaluation of a User Experience Questionnaire for Remote Labs

Abstract

Remote laboratories have been in use for 25 years now. Whereas several learning oriented meta-analyses exist, validated and agreed-upon tools for assessing the user experience are not readily available. The present paper fills this gap by designing and evaluating a questionnaire focused on the needs of remote labs developers and educators using them. Building from pre-existing tools, a first version of the User eXperience Questionnaire (UXQ version 20190308) was designed to contain four scales, usability, utility, satisfaction and immersion. A total of 180 completed responses were collected from two different remote labs (VISIR and FPGA), in different campuses and in different courses to evaluate the questionnaire. The questionnaire was analyzed in terms of reliability, using Cronbach's alpha and McDonalds' omega, and validity of construct, through factor analysis. The reliability of the questionnaire and its four subscales is acceptable, but its validity should be improved. Accordingly, the questionnaire was redesigned to obtain the User eXperience Questionnaire (UXQ version 20191126), which includes three scales: usability, utility and immersion, and three questions per scale. This questionnaire was assessed using the same data. Reliability coefficients are above 0.7 and construct validity is satisfactory. A new questionnaire to evaluate the user experience in remote laboratories has been designed and validated. The questionnaire, now renamed as UXQ4RL v. 1.0, is presented and made available in this paper.

Development of Questionnaire to Evaluate Students’ Perception about Real and Virtual Refutational Laboratory: A Rasch Measurement Approach

“…The output of this questionnaire can be used as a reference by researchers to find out students' views related to experiences after participating in learning using real and virtual RF methods. This instrument set can also be used by teachers in carrying out their work to evaluate the learning process carried out in order to improve the quality of their learning (Cuadros et al, 2021;Lutasari & Kartowagiran, 2019).…”

“…Therefore in conducting research or learning, researchers or teachers need instruments to evaluate the implementation of remediation of misconceptions that use the refutational laboratory method as a form of educational improvement quality (Cuadros et al, 2021;Lutasari & Kartowagran, 2019).…”

A Tool to Assess Architectural Education from the Sustainable Development Perspective and the Students’ Viewpoint

“…On the one hand, to assess the items individually, the proportion of chosen answers was analyzed. On the other hand, to assess the reliability of the overall questionnaire, the internal consistency of the scales was measured through the Cronbach's alfa indicator [130][131][132][133][134][135][136][137], in line with the reliability assessment conducted with the questionnaires used as a reference [19,77]. Additionally, the proportion of "Don't know" answers were assessed [101].…”

“…On the other hand, the reliability of the final version of the questionnaire was assessed by analyzing the internal consistency of its scales through the Cronbach's alpha coefficient, according to different literature [130][131][132][133][134][135][136][137] and to the reliability assessment performed with the questionnaires used as a reference [19,77]. This measure indicates the extent to which the different items of a scale are closely related as a group.…”

“…This measure indicates the extent to which the different items of a scale are closely related as a group. In particular, values of 0.7 or higher are considered acceptable [130][131][132][133], although some authors also refer to acceptable values of 0.6 or higher [62,[138][139][140].…”