Do Examinees Have Similar Test-Taking Effort? A High-Stakes Question for Low-Stakes Testing

Barry, Carol L.; Horst, S. Jeanne; Finney, Sara J.; Brown, Allison R.; Kopp, Jason P.

doi:10.1080/15305058.2010.508569

Cited by 79 publications

(55 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, the low stakes have been shown to lead to reduced validity and reliability when stu dents are not putting their best effort into the assessment because of the lack of personal consequences. They have also led to underestimated norms in later high-stakes tests on the basis of the results (Barry, Horst, Finney, Brown, & Kopp, 2010;Wise, 2006). The problem of low stakes has been countered by adding to the assessments self-report questions regarding the effort students have invested in the assessment, but the reliability of self-reported effort has been shown to be unreliable because of untruthful responding and not accounting for change in effort along an entire test session (cf.…”

mentioning

confidence: 95%

The role of time on task in computer-based low-stakes assessment of cross-curricular skills.

Kupiainen

Vainikainen

Marjanen

et al. 2014

Journal of Educational Psychology

View full text Add to dashboard Cite

The role of time on task (TOT) for students' attainment in a low-stakes assessment of cross-curricular skills was examined using the log data collected in the computer-based assessment (CBA). Two structural equation models were compared: Model 1, in which students' test scores were explained by grade point average (GPA) together with mastery and detrimental motivational attitudes, and Model 2, in which TOT was added to the model to mediate the effects of GPA and the 2 motivational constructs. Fitting the models to nationally representative data of 4,249 Finnish 9th graders (Mage = 15.92 years) confirmed the hypothesis that investment of time plays a key role in explaining test scores in low-stakes assessment even when prior ability (GPA) is taken into account. It was also confirmed that the effects of the detrimental attitudes on students' attainment were mediated by TOT. The study makes an important contribution to research regarding the role of motivational attitudes and TOT in low-stakes assessment, which is vital for the use of the assessment results in national and international benchmarking. It is concluded that log data provide a functional way to investigate time investment in CBA as an indicator for students' effort, yielding relevant implications for educational psychologists.

show abstract

mentioning

confidence: 95%

The role of time on task in computer-based low-stakes assessment of cross-curricular skills.

Kupiainen

Vainikainen

Marjanen

et al. 2014

Journal of Educational Psychology

View full text Add to dashboard Cite

show abstract

“…Examinee effort is on average not only lower in low-stakes assessments compared to high-stakes assessments, but it is also likely to be more variable (Barry et al, 2010). Because examinee effort is related to test performance, and low examinee effort tends to result in a distorted ability estimate, the exerted effort can be a source of construct-irrelevant variance (Haladyna & Downing, 2004 ).…”

Section: Examinee Elfort In Low-stakes Assessmentsmentioning

confidence: 99%

“…This rnight cause sorne test takers to expend low effort during the assessrnent, which can result in biased and invalid rneasurernents. Therefore, a high-stakes question arises for low-stakes assessrnents (Barry, Horst, Finney, Brown, & Kopp, 2010): Does exarninee effort-and the differences therein-forrn a threat for the validity ofinternational assessrnents?…”

mentioning

confidence: 99%

Student, School, and Country Differences in Sustained Test-Taking Effort in the 2009 PISA Reading Assessment

Debeer

Buchholz²,

Hartig

et al. 2014

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

In this article, the change in examinee effort during an assessment, which we will refer to as persistence, is modeled as an effect of item position. A multilevel extension is proposed to analyze hierarchically structured data and decompose the individual differences in persistence. Data from the 2009 Program of International Student Achievement (PISA) reading assessment from N ¼ 467,819 students from 65 countries are analyzed with the proposed model, and the results are compared across countries. A decrease in examinee effort during the PISA reading assessment was found consistently across countries, with individual differences within and between schools. Both the decrease and the individual differences are more pronounced in lower performing countries. Within schools, persistence is slightly negatively correlated with reading ability; but at the school level, this correlation is positive in most countries. The results of our analyses indicate that it is important to model and control examinee effort in low-stakes assessments.

show abstract

“…In fact, three students chose not to take this posttest. Research conducted by Barry et al (2010) shows that test scores may not truly reflect the abilities of examinees in cases when there is little or no personal consequence for the test taker.…”

Section: Pre-and Postcourse Knowledge Assessmentmentioning

confidence: 99%

Enhancing Geoscience Education Within a Minority-Serving Preservice Teacher Population

Ellins

Olson

2012

Journal of Geoscience Education

View full text Add to dashboard Cite

The University of Texas Institute for Geophysics and Huston-Tillotson University collaborated on a proof of concept project to offer a geoscience course to undergraduate students and preservice teachers in order to expand the scope of geoscience education within the local minority student and teacher population. Students were exposed to rigorous Earth science materials, geoscientists conducting cutting-edge research, headliner topics, and pedagogical approaches to teaching. An evaluation of the data reveal that the course received mixed, but overall positive reviews and that student performance was mixed. Pre-and posttest results indicate that students made only modest gains. Half of the students performed at levels that matched our expectations and will be able to apply the geoscience knowledge and skills that they learned in an elementary school setting. The course contributed to the preparation of minority teachers to teach Earth science in Texas, filling a critical need. The authors, in collaboration with a minority-serving institution and as part of the preparation for a preservice teacher program, benefited from the experience; they subsequently applied the lessons learned to a program of professional development for minority-serving science teachers, the

show abstract

Do Examinees Have Similar Test-Taking Effort? A High-Stakes Question for Low-Stakes Testing

Cited by 79 publications

References 26 publications

The role of time on task in computer-based low-stakes assessment of cross-curricular skills.

The role of time on task in computer-based low-stakes assessment of cross-curricular skills.

Student, School, and Country Differences in Sustained Test-Taking Effort in the 2009 PISA Reading Assessment

Enhancing Geoscience Education Within a Minority-Serving Preservice Teacher Population

Contact Info

Product

Resources

About