How characteristics of student essays influence teachers' evaluations.

Freedman, Sarah Warshauer

doi:10.1037/0022-0663.71.3.328

Cited by 78 publications

(49 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Eckes, 2012Eckes, , 2015Engelhard, 1994;Lumley, 2002;Myford & Wolfe, 2009;Weigle, 1998) eller vurdering relatert til tekstlingvistiske trekk (Freedman, 1979;Fritz & Ruegg, 2013;Wolfe, Song, & Jiao, 2016;Xie, 2015;Ö stlund-Stjärnegårdh, 2002). Vår studie har eit anna utgangspunkt: når oppgåver er sjangerlause, vil ein føresetnad for at vurderingane ikkje skal sprike, vere at ein møter tekstar med same lesemåte.…”

Section: Gb Skar and Aj Aasenunclassified

Risikotrekk og skjulte kvalitetar i elevtekstar

Skar¹,

Aasen²

2016

NJLR

View full text Add to dashboard Cite

Abstract*norskDenne artikkelen undersøker ein elevtekst frå dei nasjonale utvalsprøvene i skriving som ei rekkje ekspertvurderarar har bedømt svaert ulikt. Formålet med analysen er å avdekke vurderingsområde som kan vere krevjande å vurdere når oppgåver ikkje spesifiserar kva sjanger elevane skal skrive. Analysen viser at ein elevtekst kan lesast på ulike måtar, som anten sakprega eller litteraer tekst. Valet av lesemåte verkar inn på kor vidt teksten møter kvalitetskriterium, altså om han blir vurdert som god eller ikkje. Relevansen av ein slik analyse ligg i at denne typen kunnskap har implikasjonar for utvikling av skriveoppgåver, vurderingskriterier og vurderingsrettleiingar.Nøkkelord: Vurdering; vurdering av skriving; tekstanalyse; skriveprøve Abstract*engelskThis article presents an investigation of a student script that has received highly different ratings by a group of expert raters. The purpose of the analysis is to shed light on rating scales that might be difficult to use when prompts in a writing test do not specify which genre the student is supposed to use. The results show that it is possible to read the specific script in two ways, either as fiction or as non-fiction. Choosing to read it in a particular way effects the interpretation of criteria associated with different band levels on different scales. The results are relevant for all working with writing test development, including teachers designing classroom assessment tasks.

show abstract

Section: Gb Skar and Aj Aasenunclassified

Risikotrekk og skjulte kvalitetar i elevtekstar

Skar¹,

Aasen²

2016

NJLR

View full text Add to dashboard Cite

show abstract

“…The danger is not always realized; research by Page and by Quellmalz et al (1982) shows that individual multitrait ratings can produce unique variance. On the other hand, research by Sarah Freedman (1979) demonstrates the powerful associative, or halo, effect that can undermine the validity of analytic subscores. Chi-square analysis revealed significant correlations of "content" with "organization" and of "sentence structure" with "mechanics" in a group of essays.…”

Section: Scoring Methods For Direct Assessmentmentioning

confidence: 99%

The Assessment of Writing Ability: A Review of Research

Cooper¹

1984

ETS Research Report Series

View full text Add to dashboard Cite

The assessment of writing ability has recently recevied much attention from educators, legislators, and measurement experts, especially because the writing of students in all disciplines and at all educational levels seems, on the whole, less proficient than the writing produced by students five or ten years ago. The GRE Research Committee has expressed interest in the psychometric and practical issues that pertain to the assessment of writing ability. This paper presents not a new study but a review of major research in light of GRE Board concerns. Specifically, recent scholarship and information from established programs are used to investigate the nature and limitations of essay and multiple-choice tests of writing ability, the statistical relationship of performances on these types of tests, the performance of population subgroups on each kind of task, the possible need of different disciplines for different tests of composition skill, and the cost and usefulness of various strategies for evaluating writing ability.The literature indicates that essay tests are often considered more valid than multiple-choice tests as measures of writing ability. Certainly they are favored by English teachers. But although essay tests may sample a wider range of composition skills, the variance in essay test scores can reflect such irrelevant factors as speed and fluency under time pressure or even penmanship. Also, essay test scores are typically far less reliable than multiple-choice test scores. When essay test scores are made more reliable through multiple assessments, or when statistical corrections for unreliability are applied, performance on multiple-choice and essay measures can correlate very highly. The multiple-choice measures, though, tend to overpredict the performance of minority candidates on essay tests. It is not certain whether multiple-choice tests have essentially the same predictive validity for candidates in different academic disciplines, where writing requirements may vary. Still, at all levels of education and ability, there appears to be a close relationship between performance on multiple-choice and essay tests of writing ability. And yet each type of measure contributes unique information to the overall assessment. The best measures of writing ability have both essay and multiple-choice sections, but this design can be prohibitively expensive. Cost cutting alternatives such as an unscored or locally scored writing sample may compromise the quality of the essay assessment. For programs considering an essay writing exercise, a discussion of the cost and uses of different scoring methods is included. The holistic method, although having little instructional value, offers the cheapest and best means of rating essays for the rank ordering and selection of candidates.

show abstract

“…However, the raters in Freedman's (1979) study said that organization was the second most difficult construct to assess with the content construct being the most difficult. Although raters find organization to be important, assessing and measuring organization is a challenge.…”

Section: Literature Review Defining Organizationmentioning

confidence: 99%

“…There was a tendency for teachers to provide more feedback on content and organization than sentence structure and mechanics in the L1 setting. Freedman (1979) also conducted a study on raters' perceptions of a piece of academic writing and found that the rewriting of essays to be stronger in content and organization had a greater impact on scores in comparison to those that were rewritten to be stronger in sentence structure and mechanics. It can be concluded from these studies that raters consider organization to be of great importance in academic essays.…”

Section: Literature Review Defining Organizationmentioning

confidence: 99%

See 1 more Smart Citation

Organization of ideas in writing: what are raters sensitive to?

Ruegg

Sugiyama

2013

Language Testing in Asia

View full text Add to dashboard Cite

Whether foreign language writing is rated using analytic rating scales or holistically, the organization of ideas is invariably one of the aspects assessed. However, it is unclear what raters are sensitive to when rating writing for organization. Which do they value more highly, the physical aspects of organization, such as paragraphing and the existence of organization markers, or deeper textual aspects, such as the coherent flow of ideas? This study investigates whether raters of timed essays value paragraphing, cohesive devices or coherence more when assigning a score for organization. The current study used multiple regression to ascertain what raters of the writing section of an in-house proficiency test were sensitive to when rating writing for organization using an analytic rating scale. The number of paragraphs, number of cohesive devices and coherence within 116 timed essays were evaluated and it was found that raters seem to value the physical aspects of organization more than the deeper textual aspects. Specifically, the number of paragraphs and the number of cohesive devices predicted differences in scores assigned for organization. In addition, the scores assigned for content were significantly predictive of scores for organization.

show abstract

How characteristics of student essays influence teachers' evaluations.

Cited by 78 publications

References 12 publications

Risikotrekk og skjulte kvalitetar i elevtekstar

Risikotrekk og skjulte kvalitetar i elevtekstar

The Assessment of Writing Ability: A Review of Research

Organization of ideas in writing: what are raters sensitive to?

Contact Info

Product

Resources

About