The Effects of Motivational Instruction on College Students' Performance on Low-Stakes Assessment

Liu, Ou Lydia; Rios, Joseph A.; Borden, Victor M. H.

doi:10.1080/10627197.2015.1028618

Cited by 57 publications

(50 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To put this finding into context, this difference in performance matches the performance differences typically found between freshmen and seniors (Blaich and Wise 2011;Arum and Roksa 2014). Previous research has found similar differences in performance between motivated and unmotivated examinees (e.g., Wise and Kong 2005;Wise and DeMars 2010;Liu et al 2012;Rios et al 2014;Liu et al 2015). These results stress the importance of considering student effort or motivation on low-stakes SLO measures.…”

Section: Student Effortsupporting

confidence: 50%

“…In fact, upon further analyses, results showed that of the 114 students who indicated perceived that they did not have enough time, 75% of them indicated that they tried their best, which could suggest some relationship between testing time and student effort. Liu et al (2015) found that on average, motivated students spent 15 seconds longer on individual test items as compared to unmotivated students. Because higher motivated students may take longer on test items, they may have also felt as though there wasn't enough time to complete the assessment items.…”

Section: Student Effortmentioning

confidence: 99%

“…We should also investigate how response time information can be used with self-report data to effectively identify motivated and unmotivated students. Additionally, future research should also evaluate methods to improve student motivation at the start of the assessment through methods such as changes in the instructions for the assessment, which have been found to be an effective method for motivating students (see Liu et al 2012;Liu et al 2015).…”

Section: Student Effortmentioning

confidence: 99%

See 2 more Smart Citations

Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy

Roohr¹,

Lee²,

Xu³

et al. 2017

Numeracy

View full text Add to dashboard Cite

Quantitative literacy has been identified as an important student learning outcome (SLO) by both the higher education and workforce communities. This paper aims to provide preliminary evidence of the psychometric quality of the pilot forms for HEIghten quantitative literacy, a next-generation SLO assessment for students in higher education. We evaluated the psychometric quality of the test items (e.g., item analyses), individual-and group-level reliability, the relationship with student performance and related variables (e.g., grade point average) as well as student perceptions, and differences across college-related and demographic subgroups. Our study used data from a pilot test administered to over 1,500 students at 23 higher education institutions in the United States. Results showed that (a) overall, items were functioning well, but a small portion of items should be dropped due to unsatisfactory performance; (b) correlations across sub-areas of the assessment were very high indicating that the assessment may be unidimensional; (c) reliability estimates similar to existing SLO assessments were found at both individual and group levels; (d) assessment scores correlated positively with high school and college GPA, number of math college courses, self-rated quantitative literacy skills, and college admissions scores; (e) students had positive perceptions about the assessment; and (f) performance differences were found across institution type, college majors, gender, racial/ethnic groups, and language groups, but not across credit-hour categories. Implications for operational test development and understanding of quantitative literacy performance are discussed. Zhen Wang is a Senior Psychometrician at the Educational Testing Service (ETS). She holds a Ph.D. in Educational Measurement and Research Methodology from the University of British Columbia. Currently, she is the statistical coordinator for ETS' HEIghten® Outcomes Assessment Suite, Major Field Test, Success Navigator, Proficiency Profile, and iSkills projects. She also works on the research projects related to automated scoring and higher education. Her research interests include item response theory, structural equation models, hierarchical linear models, rater models, and equating and scaling techniques.

show abstract

Section: Student Effortsupporting

confidence: 50%

Section: Student Effortmentioning

confidence: 99%

Section: Student Effortmentioning

confidence: 99%

See 1 more Smart Citation

Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy

Roohr¹,

Lee²,

Xu³

et al. 2017

Numeracy

View full text Add to dashboard Cite

show abstract

“…Consequently, as this investigation and others have relied upon adult volunteers (although perhaps not Liu et al, 2015), the vast majority of which appear to have been at least neutral test-taking motivation, it may be that incentives would necessarily be ineffective at increasing intelligence test performance. Consequently, as this investigation and others have relied upon adult volunteers (although perhaps not Liu et al, 2015), the vast majority of which appear to have been at least neutral test-taking motivation, it may be that incentives would necessarily be ineffective at increasing intelligence test performance.…”

Section: Non-experimental Resultsmentioning

confidence: 92%

“…That is, even in the condition for which there was no financial incentive to increase intelligence test performance, only 13% of the testees scored a mean item response of <3.0 on the Effort subscale (theoretical range: 1.0-5.0). Importantly, once the testees who exhibited extremely low test-taking motivation were removed from the Liu et al (2015) sample, 2 the incentive and non-incentive group academic achievement means were not found to differ statistically significantly. Consequently, it may be suggested that Liu et al's (2015) sample was atypically unmotivated to complete their cognitive ability-type testing, in comparison with other investigations that have administered the Student Opinion Scale to university students (e.g., Thelk et al, 2009; Effort subscale mean = 3.44).…”

Section: Incentives and Intelligence Test Scoresmentioning

confidence: 98%

A moderate financial incentive can increase effort, but not intelligence test performance in adult volunteers

Gignac

2018

British J of Psychology

View full text Add to dashboard Cite

A positive correlation between self-reported test-taking motivation and intelligence test performance has been reported. Additionally, some financial incentive experimental evidence suggests that intelligence test performance can be improved, based on the provision of financial incentives. However, only a small percentage of the experimental research has been conducted with adults. Furthermore, virtually none of the intelligence experimental research has measured the impact of financial incentives on test-taking motivation. Consequently, we conducted an experiment with 99 adult volunteers who completed a battery of intelligence tests under two conditions: no financial incentive and financial incentive (counterbalanced). We also measured self-reported test-taking importance and effort at time 1 and time 2. The financial incentive was observed to impact test-taking effort statistically significantly. By contrast, no statistically significant effects were observed for the intelligence test performance scores. Finally, the intelligence test scores were found to correlate positively with both test-taking importance (r = .28) and effort (r = .37), although only effort correlated uniquely with intelligence (partial r = .26). In conjunction with other empirical research, it is concluded that a financial incentive can increase test-taking effort. However, the potential effects on intelligence test performance in adult volunteers seem limited.

show abstract

Measuring Motivation in Low‐Stakes Assessments

Finn

2015

ETS Research Report Series

View full text Add to dashboard Cite

There is a growing concern that when scores from low‐stakes assessments are reported without considering student motivation as a construct of interest, biased conclusions about how much students know will result. Low motivation is a problem particularly relevant to low‐stakes testing scenarios, which may be low stakes for the test taker but have considerable consequences for teachers, school districts, or educational and governmental institutions. The current review addresses the impact of motivation on assessment scores and research that have identified methods for minimizing the error introduced by unmotivated test takers. A comprehensive review of the test conditions that have been shown to influence motivation is provided. In addition, the review identifies a number of circumstances under which motivation can be enhanced or diminished. The benefits and limitations of the various measurement techniques that have been used to mitigate the negative impact of low‐motivation test takers on score interpretation are discussed.

show abstract

The Effects of Motivational Instruction on College Students' Performance on Low-Stakes Assessment

Cited by 57 publications

References 22 publications

Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy

Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy

A moderate financial incentive can increase effort, but not intelligence test performance in adult volunteers

Measuring Motivation in Low‐Stakes Assessments

Contact Info

Product

Resources

About