Motivation Filtering on a Multi-Institution Assessment of General College Outcomes

Steedle, Jeffrey T.

doi:10.1080/08957347.2013.853072

Cited by 14 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two important assumptions are made, namely, first, that it is possible to detect the low-effort test takers and validly measure examinee effort and, second, that there is no relation between test-taking effort and the actual level of proficiency. Results show that motivation filtering increases the average test performance (Steedle, 2014;Swerdzewski et al, 2011;Wise & DeMars, 201 O; Wise, Wise, & Bhola, 2006), both when a selfquestionnaire and RTE are used to measure examinee effort. Rios, Liu, and Bridgeman (in press) showed that RTE filtering, however, bad a slightly stronger relationship with test performance.…”

Section: Dealing With Low Examinee Effortmentioning

confidence: 98%

“…The proportion of items for which a test taker is classified into solution behavior gives a test taker's RTE. Tue applicability of RTE as a measurement of examinee effort has been repeatedly demonstrated (Silm, Must, & Taeht, 2013;Steedle, 2014;Swerdzewski et al, 2011;Wise, Pastor, &Kong, 2009). The issue ofsetting the response time threshold has been addressed by Kong, Wise, and Bhola (2007).…”

Section: Debeer Et Almentioning

confidence: 99%

“…This relation between exarninee effort and perforrnance has been repeatedly found (e.g., Abdelfattah, 2010;Liu, Bridgernan, & 503 Downloaded from http:/Jjebs.aera.net at D IPF on August 10. 2016 Student, School, and Country Di.fferences Adler, 2012;Steedle, 2014;Swerdzewski et al, 2011;Waskiwicz, 2011;Wise & DeMars, 2005).…”

Section: Examinee Elfort In Low-stakes Assessmentsmentioning

confidence: 99%

“…Most research regarding the issue of low exarninee effort in low-stakes assessrnents addresses the issues of rnanipulating exarninee effort, accounting for low exarninee effort, and rneasuring exarninee effort (e.g., Steedle, 2014;Swerdzewski, Harrnes, & Finney, 2011;Waskiwicz, 2011;Wise & DeMars, 2005;Wise & Kong, 2005). In these studies, exarninee effort is cornrnonly seen as constant throughout the assessrnent.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Student, School, and Country Differences in Sustained Test-Taking Effort in the 2009 PISA Reading Assessment

Debeer

Buchholz²,

Hartig

et al. 2014

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

In this article, the change in examinee effort during an assessment, which we will refer to as persistence, is modeled as an effect of item position. A multilevel extension is proposed to analyze hierarchically structured data and decompose the individual differences in persistence. Data from the 2009 Program of International Student Achievement (PISA) reading assessment from N ¼ 467,819 students from 65 countries are analyzed with the proposed model, and the results are compared across countries. A decrease in examinee effort during the PISA reading assessment was found consistently across countries, with individual differences within and between schools. Both the decrease and the individual differences are more pronounced in lower performing countries. Within schools, persistence is slightly negatively correlated with reading ability; but at the school level, this correlation is positive in most countries. The results of our analyses indicate that it is important to model and control examinee effort in low-stakes assessments.

show abstract

Section: Dealing With Low Examinee Effortmentioning

confidence: 98%

Section: Debeer Et Almentioning

confidence: 99%

Section: Examinee Elfort In Low-stakes Assessmentsmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Student, School, and Country Differences in Sustained Test-Taking Effort in the 2009 PISA Reading Assessment

Debeer

Buchholz²,

Hartig

et al. 2014

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

show abstract

“…The first study involved high school students taking a version of the National Assessment of Educational Progress, while the second involved college students taking a test of critical thinking. Steedle () investigates strategies for adjusting results on low‐stakes performance assessments to take account of low motivation . For a general review, see Liu, Bridgeman, and Adler ().…”

Section: Introductionmentioning

confidence: 99%

Performance assessment and standardization in higher education: A problematic conjunction?

Braun

2019

Brit J of Edu Psychol

View full text Add to dashboard Cite

Background There is unrealized potential in higher education for greater use of performance assessment, particularly in support of teaching and learning: Well‐designed performance tasks can elicit evidence regarding what students know and can do with respect to complex learning objectives. At the same time, there is some pressure, at least in the United States, to widen usage of standardized assessments in order to facilitate comparisons among institutions. Aims and methods We distinguish assessments requiring extended performances of some kind from those involving other response formats. Taking student‐produced essays as a paradigmatic example, we examine the processes of assessment design and scoring as they pertain to the validity of the inferences drawn from student work products. We also briefly review the history of standardized assessments and the roles they have played in education. Further, we describe how advances in assessment theory and practice have permitted substantial relaxation in the implementation of standardization. Results Our analysis of these issues in the higher education context draws on empirical evidence regarding both faculty usage of performance tasks (particularly as a means for enhancing the construct validity of an overall assessment) and faculty resistance to the imposition of externally developed, standardized assessments. We also describe efforts undertaken in the United States to strengthen faculty expertise in assessment design, scoring, and interpretation. Conclusions We conclude that performance tasks have an important role to play in higher education, especially in view of the complexity and depth of the learning goals set for the students. For many purposes, they cannot be replaced by simpler, cheaper forms of assessment. Efforts to facilitate comparability in evaluating institutional outcomes, however, have not borne much fruit despite the substantial investments made. We argue that resources would be better directed at improving the quality of assessments developed and employed by college faculty and suggest how that might be accomplished.

show abstract

Measuring Motivation in Low‐Stakes Assessments

Finn

2015

ETS Research Report Series

View full text Add to dashboard Cite

There is a growing concern that when scores from low‐stakes assessments are reported without considering student motivation as a construct of interest, biased conclusions about how much students know will result. Low motivation is a problem particularly relevant to low‐stakes testing scenarios, which may be low stakes for the test taker but have considerable consequences for teachers, school districts, or educational and governmental institutions. The current review addresses the impact of motivation on assessment scores and research that have identified methods for minimizing the error introduced by unmotivated test takers. A comprehensive review of the test conditions that have been shown to influence motivation is provided. In addition, the review identifies a number of circumstances under which motivation can be enhanced or diminished. The benefits and limitations of the various measurement techniques that have been used to mitigate the negative impact of low‐motivation test takers on score interpretation are discussed.

show abstract

Motivation Filtering on a Multi-Institution Assessment of General College Outcomes

Cited by 14 publications

References 19 publications

Student, School, and Country Differences in Sustained Test-Taking Effort in the 2009 PISA Reading Assessment

Student, School, and Country Differences in Sustained Test-Taking Effort in the 2009 PISA Reading Assessment

Performance assessment and standardization in higher education: A problematic conjunction?

Measuring Motivation in Low‐Stakes Assessments

Contact Info

Product

Resources

About