Do outcomes from high stakes examinations taken in test centres and via live remote proctoring differ?

Cherry, Gemma; O’Leary, Michael; Науменко, Оксана Анатольевна; Kuan, Li-Ann; Waters, Linda G.

doi:10.1016/j.caeo.2021.100061

Cited by 9 publications

(15 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Exams investigated here had high reliability values and this was not affected by the manner of delivery, with both in-person and online exams having values above 0.9. High reliability for high stakes examinations taken online using remote proctoring, comparable to in-person exams, have been previously reported . An important distinction, however, is that the online exam performance discussed in this investigation is for an unproctored exam.…”

Section: Resultsmentioning

confidence: 50%

“…High reliability for high stakes examinations taken online using remote proctoring, comparable to in-person exams, have been previously reported. 26 An important distinction, however, is that the online exam performance discussed in this investigation is for an unproctored exam.…”

Section: Overall Exam Analysismentioning

confidence: 99%

“…Physical and psychological differences in the online test-taking environment are infrequently raised as potential variables influencing student performance. , Far more common are observational studies of two groups and administering comparable or identical exams and testing for differences in the overall exam performance. Although some investigations report psychometric properties of the exams, , use more advanced statistical methods, − or report aspects of test-taking behavior, such as time to complete the test, ,, most studies simply compare the overall score on the exam or the grade in the class.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Evaluating and Improving Questions on an Unproctored Online General Chemistry Exam

2022

View full text Add to dashboard Cite

Administering exams in large enrollment courses is challenging and systems in place for accomplishing this task were upended in the spring of 2020 when a sudden transformation to online instruction and testing occurred due to the COVID-19 pandemic. In the following year, when courses remained online, approaches to improve exam security included measures like using test banks and reducing the allotted time for completing exams to reduce the sharing of information. A psychometric comparison using classical test theory of an unproctored online exam with one delivered in-person indicates both have comparable reliability. However, item-level analysis demonstrates some questions performed higher in the unproctored setting, with an important variable being the item’s searchability online. Revising questions to increase generalizability and reduce searchability mitigate these performance differences. Further, changing the format for questions involving calculations from multiple-choice to open response with random number generation was found to increase difficulty and item discrimination and is a viable approach for improving exam security.

show abstract

Section: Resultsmentioning

confidence: 50%

Section: Overall Exam Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating and Improving Questions on an Unproctored Online General Chemistry Exam

2022

View full text Add to dashboard Cite

show abstract

“…The cost of setting up an online classroom being much lower than that of a brick-and-mortar classroom, many teachers have been exploring these learning spaces, making more and more useful materials available to students worldwide. Assessing and comparing the performance of remote and in-person learners as well as the effect of proctoring on student performance is still under investigation (Cherry et al, 2021;Wuthisatian, 2020). There is also hope that in the future, more advanced technologies like deep learning (Kaddoura & Gumaei, 2022) and blockchain (Sattar et al, 2023) might provide more effective, efficient and secure frameworks for assessment in online classrooms.…”

Section: Teaching and Learning In Video-conference Classroomsmentioning

confidence: 99%

Exploring Learners’ Experiences in Video-conference Classrooms through theExtended Technology Acceptance Model

Edwards

Edwards²

2022

AJEE

View full text Add to dashboard Cite

The forced switch to fully online learning during the pandemic lockdown resulted in the conversion of video-conference platforms into digital classrooms. The emergency situation at the time left no room for necessary research, training and adequate deployment process, thus forcing both learners and instructors to adopt these learning environments as the only solution to learning during those times. After the lockdown seasons were over, several educational institutions have been considering hybrid instructional modes, thereby making these platforms to persist within education. This necessitates the assessment of learners’ perception on learning in these new classrooms. This study aims to inform and inspire the design of inclusive, and sensitive future learning environments and schools where the needs of all types of learners are factored into instructional design and delivery. An analysis model based on the Technology Acceptance Model (TAM) was deployed in this study which reports on findings from an online survey of 86 respondents consisting of 48 males and 38 females all educated at post-secondary levels. Respondents ages were between 17 and 60 years with a mean age of 30.49 years. Findings show positive learners’ perceptions of video-conference in terms of most constructs, with more neutral and almost negative feedback regarding support for collaboration and instructor presence. No negative perceptions were reported regarding frustration with the technology, and no difference in perception was expressed regarding continuation of virtual learning on video-conference, or return to physical classroom. Platforms were rated by respondents based on factors considered by respondents as important for learning; the highest ratings were assigned based on features including screen sharing, host meeting control capabilities and guest control.

show abstract

“…Cherry et al (2021) examined professional licensure examinations in the USA, comparing outcomes for tests administered either using remote online proctoring or in test centres. While statistically significant differences were observed in results obtained between the two modes, no detectable pattern was observed in favour of either mode.Morin et al (2022) investigated a high-stakes national medical licensing examination in Canada taken via remote online proctoring or in exam centres.…”

mentioning

confidence: 99%

The Delivery of Speaking Tests in Traditional or Online Proctored Mode: A Comparability Study

2023

langta

View full text Add to dashboard Cite

This paper investigates the comparability of test scores recorded for high-stakes English language Speaking Tests administered face-to-face in either a traditional centre-based mode (TM) or in an online proctored mode (OLP). The data comprise a large sample of test takers taking English language Speaking Tests at four CEFR (the 'Common European Framework of Reference for Languages') levels -B1 to C2via TM or OLP. The data were analysed using descriptive statistics, effect size differences and equivalence tests. While a degree of difference in scores obtained between modes was apparent at C2 level, the differences were not found to be statistically significant. The paper concludes that whether Speaking Tests are delivered in online proctored mode or in traditional face-to-face mode, test takers receive similar scores. The study confirms that mode of test delivery does not significantly affect test taker scores.

show abstract

Do outcomes from high stakes examinations taken in test centres and via live remote proctoring differ?

Cited by 9 publications

References 24 publications

Evaluating and Improving Questions on an Unproctored Online General Chemistry Exam

Evaluating and Improving Questions on an Unproctored Online General Chemistry Exam

Exploring Learners’ Experiences in Video-conference Classrooms through theExtended Technology Acceptance Model

The Delivery of Speaking Tests in Traditional or Online Proctored Mode: A Comparability Study

Contact Info

Product

Resources

About