A Framework of the Training Module for Untrained Observers in Usability Evaluation Motivated by COVID-19: Enhancing the Validity of Usability Evaluation for Children’s Educational Games

Alshar’e, Marwan; Al‐Badi, Ali H.; Jawarneh, Malik; Tahir, Noman; Amri, Marya Al

doi:10.1155/2022/7527457

Cited by 5 publications

(7 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A framework for training the observer (see Figure 2) was proposed by [23]. e training as stipulated by the framework was a requirement to the methodology usage.…”

Section: Usability Evaluation For Children's Mobile Learningmentioning

confidence: 99%

“…e findings and recommendations of [23,37] were adapted in this study in performing the actual evaluation, with the purpose of measuring and understanding the present situation in Oman, particularly with respect to the application of imported educational games that has appropriately served the study's purpose.…”

Section: Usability Evaluation For Children's Mobile Learningmentioning

confidence: 99%

“…Culture and learning perceptions vary, and, thus, evaluation on them may generate varying outcomes. Also, observers may have varied competency level, and it is important that observer is sufficiently competent in performing the task so that the input provided in usability evaluations is reliable [23].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Usability Evaluation of Educational Games: An Analysis of Culture as a Factor Affecting Children’s Educational Attainment

Alshar’e¹,

Al‐Badi²,

Jawarneh³

et al. 2022

Advances in Human-Computer Interaction

Self Cite

View full text Add to dashboard Cite

Educational games have been employed among Omani schools but those used by local Omani schools were imported and were mostly designed based on western contexts. For Omani children, these games may be culturally inappropriate and difficult to comprehend and follow, impeding children’s learning. Three questionnaires and one observational checklist were used to gather data from 40 respondents (observers). SPSS was used in data analysis. Through experiments, the behavior of Omani students towards the use of imported educational games was examined. Five main factors, namely, efficiency, learnability, memorability, errors, and satisfaction, of educational games for a target user were measured using Hybrid User Evaluation Methodology for Remote Evaluation (HUEMRE), Training Framework for Untrained Observer (TFUO), and Framework on Educational Games Behavior Intention (EGsBI), which are specifically designed frameworks for this purpose. The results of this study explained that the Omani children are facing difficulties in using the imported educational games; furthermore, this study proves that culture, language, animation, and interaction are contributing heavily to benefiting from educational games, and, therefore, these factors shall be highly considered in the process of educational games design to facilitate and ensure children learning; furthermore, the findings of this study enrich the comprehension of how the specified factors positively affect behavioral intention of Omani students in the use of educational games and in improving the behavior intention level of these students.

show abstract

“…A framework for training the observer (see Figure 2) was proposed by [23]. e training as stipulated by the framework was a requirement to the methodology usage.…”

Section: Usability Evaluation For Children's Mobile Learningmentioning

confidence: 99%

Section: Usability Evaluation For Children's Mobile Learningmentioning

confidence: 99%

See 1 more Smart Citation

Usability Evaluation of Educational Games: An Analysis of Culture as a Factor Affecting Children’s Educational Attainment

Alshar’e¹,

Al‐Badi²,

Jawarneh³

et al. 2022

Advances in Human-Computer Interaction

Self Cite

View full text Add to dashboard Cite

show abstract

“…An important factor in software development is usability because usability can evaluate software features [33]- [35]. There are two types of usability evaluation concepts: summative (related to effectiveness, efficiency, and satisfaction) and formative (related to usability problem detection and intervention design) [36]- [38].…”

Section: Introductionmentioning

confidence: 99%

User Interface Design of Jaipong Dance Applications for Elementary School using the User-Centered Design (UCD) Method

Martha¹,

Muqorobin²,

Riskiana³

et al. 2023

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

The Jaipong dance is one of the traditional Indonesian dances, the nation's cultural heritage originating from West Java. Introducing dance to elementary school-level children is one way to maintain and preserve regional culture. However, the limited learning time causes the material to be delivered less in-depth, so students feel bored, lose motivation, and get some critical information from the material. This study aims to build a mobile-based learning application for Jaipong dance learning activities at the elementary school level using the User-Centered Design (UCD) method. The application evaluation involved fifteen public elementary school students in Ciamis, West Java. Participants were asked to answer a quantitative survey using the Quality Use Integrated Measurement (QUIM) instrument to discover their experience using the application during Jaipong dance lessons. The test results get an average score of 91%, included in the "excellent" category. These results were strengthened by the user's understanding of the Jaipong dance. As many as 86.7% of users who use the Jaipong dance learning application understand the material well. In contrast, only 33.3% of YouTube users understand the material well. This paper shows that the UCD method contributes to the design process according to the needs and characteristics of the user to design the user interface of the Jaipong dance learning application.

show abstract

“…In summary, while conducting a review of the remote usability testing literature for the broader project, we discovered studies that reported on “how to” support the moderator and observer roles [ 7 , 10 , 11 ]. Other authors [ 5 , 6 ] made a call to action to better comprehend complex linkages among human factors (eg, social context or environment or “who” is present at testing), human-computer interaction, and web-based technology and tools to advance the remote usability testing approach.…”

Section: Introductionmentioning

confidence: 99%

Remote Moderator and Observer Experiences and Decision-making During Usability Testing of a Web-Based Empathy Training Portal: Content Analysis

Lobchuk¹,

Bathi²,

Ademeyo³

et al. 2022

JMIR Form Res

View full text Add to dashboard Cite

Background COVID-19 restrictions severely curtailed empirical endeavors that involved in-person interaction, such as usability testing sessions for technology development. Researchers and developers found themselves using web-based moderation for usability testing. Skilled remote moderators and observers are fundamental in this approach. However, to date, more empirical work is needed that captures the perceptions and support needs of moderators and observers in testing situations. Objective The aim of this paper was to identify remote moderator and observer participant experiences and their use of certain tools to capture feedback of users as they interact with the web browser application. Methods This research is part of a broader study on an educational web browser application for nursing students to learn perspective taking and enhance their perceptual understanding of a dialogue partner’s thoughts and feelings. The broader study used a quantitative and think-aloud qualitative problem-discovery usability study design. This case study explored written accounts of the remote moderator and observer participants regarding their roles, experiences, and reactions to the testing protocol and their suggestions for improved techniques and strategies for conducting remote usability testing. Content analysis was used to analyze participants’ experiences in the usability testing sessions. Results We collected data from 1 remote moderator and 2 remote observers. Five themes were identified: dealing with personal stressors, dealing with user anxiety, maintaining social presence, ethical response to the study protocol, and communication during sessions. The participants offered recommendations for the design of future remote testing activities as well as evidence-informed training materials for usability project personnel. Conclusions This study’s findings contribute to a growing body of endeavors to understand human-computer interaction and its impact on remote moderator and observer roles. As technology rapidly advances, more remote usability testing will occur where the knowledge gleaned in this study can have an impact. Recommendations based on moderator and observer participant perspectives identify the need for more evidence-informed training materials for their roles that focus on web-based interpersonal communication skills, execution of user testing protocols, troubleshooting technology and test user issues, proficiency in web conferencing platforms, behavior analysis and feedback technologies, and time management.

show abstract

A Framework of the Training Module for Untrained Observers in Usability Evaluation Motivated by COVID-19: Enhancing the Validity of Usability Evaluation for Children’s Educational Games

Cited by 5 publications

References 46 publications

Usability Evaluation of Educational Games: An Analysis of Culture as a Factor Affecting Children’s Educational Attainment

Usability Evaluation of Educational Games: An Analysis of Culture as a Factor Affecting Children’s Educational Attainment

User Interface Design of Jaipong Dance Applications for Elementary School using the User-Centered Design (UCD) Method

Remote Moderator and Observer Experiences and Decision-making During Usability Testing of a Web-Based Empathy Training Portal: Content Analysis

Contact Info

Product

Resources

About