Comparison of test performance on paper-based testing (PBT) and computer-based testing (CBT) by English-majored undergraduate students in China

Abstract

Computer-based testing (CBT), which refers to delivering assessments with computers, has been widely used in large English proficiency tests worldwide. Despite an increasing CBT in China, limited research is available concerning whether CBT can be used for the Test for English Majors-Band 4 (TEM 4). The current study investigated whether testing mode impacted TEM 4 score and factors (i.e., computer familiarity level and attitude towards CBT) that might correlate with performance on CBT of TEM 4. Overall 92 Chinese undergraduate students were randomly assigned to one of the groups, i.e., CBT or paper-based testing (PBT), and took the test. A mixed method was employed, including (1) quantitative and qualitative analysis of test performance in two modes, as well as CBT group participants’ computer familiarity and attitudes towards the mode; and (2) thematic analysis of semi-structured interviews. The results revealed that (1) test scores in CBT and PBT were comparable; (2) two items in the computer familiarity questionnaire, i.e., comfort level of reading articles on the computer and forgetting time when using computers, positively correlated with CBT scores; and (3) participants’ attitude towards CBT did not impact test performance.

Achievement Emotions in Paper-Based Exams vs. Computer-Based Exams

“…Additionally, no significant differences were found in test scores between the participants of the experimental and the controlled groups. Likewise, Yu and Iwashita (2021) randomly assigned 92 Chinese undergraduate students to an experimental group using CBT, and a controlled group using PBT. The test scores were compared, and semi-structured interviews (Pekrun, 2006).…”

“…These studies such as Öz and Özturan (2018) and Yu and Iwashita (2021) showed that computerbased testing can be integrated into university education without fears regarding test validity, reliability, or scores. However, the area that still needs further exploration is students' acceptance of this mode of testing (Zheng & Bender, 2019).…”

Students’ Attitude Towards Ruanguji Mobile-Based Assessment: An Explanatory Case Study in Tangerang, Indonesia

“…It is because students who do tests on screen have a variety of concerns and problems on computer-based examinations, such as noisy keyboards, assessment of spelling, and unfairness rather than those who are less comfortable with the technology (Yeom, Jun, 2020). In addition, research by W. Yu and N. Iwashita (Yu, Iwashita, 2021) indicates that the students' attitude towards MBA does not have a positive impact on their performance. Complementarily, in their research H. Öz and T. Özturan (Öz, Özturan, 2018) show that there is not any significant difference in test scores between the participants who take the computer-based test and those who take the paper-based test.…”

Validity of score interpretations on an online English placement writing test

“…Test-takers’ perceptions of and preferences for computerized testing are commonly measured aspects of computer familiarity that have drawn much attention in cross-modality studies. According to the literature, an increasing number of test-taker populations at a whole-group level positively perceive or prefer CBW assessment for a range of reasons, including convenience and efficiency of the test and confidence in and adeptness at typing (e.g., Feng et al, 2019 ; Kim et al, 2019 ; Lee, 2004 ; Yu & Iwashita, 2021 ).…”

“…A related area that has also been popularly explored in many questionnaire-based studies is the effect of test-takers’ computer familiarity on CBW test scores at a whole-group level; yet, the end result is inconclusive due to mixed findings. Some studies have revealed that the computer usage-related factor positively affects test scores or has interaction effects with computer use experiences on test scores (e.g., Breland et al, 2005 ; Lee, 2004 ; Yu & Iwashita, 2021 ). According to Jin and Yan ( 2017 ), test-takers with high levels of computer familiarity performed better on the computer version of the College English Test (CET) in China.…”