Prepublishing the Topics for a Test of Writing Skills: A Small-Scale Simulation

Powers, Donald E.; Fowles, Mary E.; Farnum, Marisa

doi:10.1207/s15324818ame0602_2

Cited by 6 publications

(15 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Analyses revealed a negligible effect as a result of having seen essay topics before the test was administered-an effect that was virtually the same as that noted previously (Powers et al, 1993). As the researchers pointed out, however, the consequences of test disclosure cannot be determined definitively outside the context of a fully operational testing program.…”

Section: Prior Researchsupporting

confidence: 60%

“…Test-takers also seem to believe that their writing skills are assessed more accurately when they are permitted to write on topics that have been considered beforehand (Powers, Fowles, & Farnum, 1993;Powers & Fowles, 1998). As a research study participant once suggested to us, prepublishing prompts should elicit writing that is "more consistent with [the kind of writing] that you would see in class.…”

Section: Rationale For Prepublishing Essay Promptsmentioning

confidence: 99%

“…Research on its effects, however, has been relatively sparse. In addition, with the exception of three studies (Hale, Angelis, & Thibodeau, 1983;Powers et al, 1993;Powers & Fowles, 1998), apparently all of the published research has addressed the impact of releasing test items after a test is administered. 1 Even the exceptions that involved prepublication are less than definitive, however, because none of these studies was conducted in a high-stakes testing environment where test scores actually counted.…”

Section: Prior Researchmentioning

confidence: 99%

“…Before The Praxis Series™ writing assessment became operational, Powers et al (1993) conducted a small-scale simulation to estimate the likely impact of disclosing essay topics. At four colleges, writing instructors were asked to take a small set of topics and, using any tricks they could muster, coach students to take the assessment.…”

Section: Prior Researchmentioning

confidence: 99%

See 3 more Smart Citations

Effects of Preexamination Disclosure of Essay Prompts for the Gre Analytical Writing Assessment

Powers

2005

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

This study examined how the practice of prepublishing prompts used on the writing section of the Graduate Record Examinations ® (GRE ® ) General Test impacts test-preparation behavior, test performance, test validity, and examinee perceptions of the value of prompt prepublication.Researchers imposed modest experimental control over how participants used the prompts to prepare for an upcoming test. The strategy test-takers reported using most frequently was simply to "think generally about the potential topics." Slightly fewer than half of study participants wrote sample essays to prepare for the test, and very few (4%) admitted to memorizing essays that could be recalled during testing. Results provided no indication that participants benefited from encountering a prompt for which they had prepared. The vast majority of study participants, however, thought that making the GRE essay prompts available ahead of time is a good testing policy.

show abstract

Section: Prior Researchsupporting

confidence: 60%

Section: Rationale For Prepublishing Essay Promptsmentioning

confidence: 99%

Section: Prior Researchmentioning

confidence: 99%

Section: Prior Researchmentioning

confidence: 99%

See 2 more Smart Citations

Effects of Preexamination Disclosure of Essay Prompts for the Gre Analytical Writing Assessment

Powers

2005

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

show abstract

“…To the extent that test takers have more time to think about the topics and to formulate ideas for developing their essay responses, disclosure holds promise, some researchers feel, for making writing tests less a reflection of the ability to extemporize and more an indication of "genuine" writing proficiency. Generally, test takers also seem to believe that writing on topics about which they have previously thought constitutes a better test of their writing skills than does writing on topics that they have not considered beforehand (Powers, Downloaded by [Australian National University] at 21:44 04 June 2016 Fowles, & Farnum, 1993). Many cognitive psychologists may concur with this I view.…”

Section: Test Preparation and Test Disclosurementioning

confidence: 99%

Effects of Preexamination Disclosure of Essay Topics

Powers¹,

Fowles²

1998

Applied Measurement in Education

Self Cite

View full text Add to dashboard Cite

To determine the effects on test performance and test validity of releasing essay topics before a writing examination, researchers asked 300 prospective graduate students to write 2 essays, 1 on a previously unseen topic and 1 on a topic that was made available about 2 weeks before the test. Although for each of the 4 essay topics studied, performance was slightly better when a topic had been disclosed, overall analyses did not reveal any statistically significant effect of disclosure on test performance. Consequently, there was no detectable effect of disclosure on the relation of test performance to several independent indicators of writing skill. Limitations of the study are discussed in terms of its generalizability to a high-stakes testing situation and the extent to which it addressed both the positive and the negative effects of disclosure.A prevailing view among composition specialists is that writing is a process, one that entails complementary activities of prewriting and planning, drafting, writing, and revising (these activities are not, however, a neat sequence of stages; Flower &Hayes, 1980). Because traditional tests of writing usually allow enough time only for developing a first draft and not for any significant planning or revision, they may not adequately elicit all of the processes that writers typically employ, and therefore, they may not fully represent all of the important facets of writing proficiency. In other words, the test may suffer from a major source of invalidity-what Messick (1989) has termed construct underrepresentation. In more lay terminology, the kind of impromptu writing that is typically required for direct measures of writing ability does not represent real writing. As one graduate faculty Requests for reprints should be sent to Donald E. Powers, Mail Stop 17-R, Educational Testing Service, Rosedale Road, Princeton, NJ 08541.

show abstract

Effects of Disclosing Essay Topics for a New Gre Writing Test

Powers

Fowles

1996

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

To determine the effects on test performance and test validity of releasing GRE essay topics before an examination, some 300 prospective graduate students each wrote two essays, one on a previously unseen topic and one on a topic that was made available about two weeks before the test. Although for each of the four essay topics studied, performance was slightly better when a topic had been disclosed, analyses did not reveal any statistically significant effect of disclosure on test performance. Nor was there any detectable effect of disclosure on the meaning of test scores, as reflected in the relationship of test performance to several independent indicators of writing skill. Limitations of the study are discussed in terms of its generalizability to a high‐stakes testing situation and the extent to which it addressed both the positive and the negative effects of disclosure.

show abstract

Prepublishing the Topics for a Test of Writing Skills: A Small-Scale Simulation

Cited by 6 publications

References 11 publications

Effects of Preexamination Disclosure of Essay Prompts for the Gre Analytical Writing Assessment

Effects of Preexamination Disclosure of Essay Prompts for the Gre Analytical Writing Assessment

Effects of Preexamination Disclosure of Essay Topics

Effects of Disclosing Essay Topics for a New Gre Writing Test

Contact Info

Product

Resources

About