Procedures and Criteria for Evaluating Reading and Listening Comprehension Tests

Marks, Edmond; Noll, Gary A.

doi:10.1177/001316446702700210

Cited by 16 publications

(5 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such procedures have been used by a few careful investigators (Beighley, 1952(Beighley, , 1954Fairbanks, Guttman, and Miron, 1957a). Marks and Noll (1967) present a technique that is to be highly recommended for evaluating items on reading and listening tests. By using the controls that they suggest, one can be reasonably certain that responses to comprehension items validly measure the degree to which the subject has been able to acquire new knowledge through exposure to verbal material.…”

Section: "mentioning

confidence: 99%

Learning From Verbal Discourse in Educational Media: A Review of the Literature

Carroll¹

1971

ETS Research Bulletin Series

View full text Add to dashboard Cite

Section: "mentioning

confidence: 99%

Learning From Verbal Discourse in Educational Media: A Review of the Literature

Carroll¹

1971

ETS Research Bulletin Series

View full text Add to dashboard Cite

“…The framework described above is similar to a number of models proposed by various authors to describe tests (e.g., Brownless & Keats, 1958;Knapp, 1977;Macready & Dayton, 1977;Marks & Noll, 1967;Wilcox, 1979b). Macready and Dayton (1977, p. 100) imply that their model is appropriate when mastery of a skill is an all-or-none process.…”

Section: Some Definitionsmentioning

confidence: 98%

An Approach to Measuring the Achievement or Proficiency of an Examinee

Wilcox

1980

Applied Psychological Measurement

View full text Add to dashboard Cite

Various school systems are developing proficiency tests which are conceptualized as representing a variety of skills with one or more items per skill. This paper discusses how certain recent technical advances might be extended to examine these tests. In contrast to previous analyses, errors at the item level are included; and it is shown that inclusion of these errors implies that a substantially longer test might be needed. One approach to this problem is described, and directions for future research are suggested.

show abstract

“…There are two different but highly related approaches that have been considered, based on the framework just described. The first, which seems to have received the most attention in the literature, is to consider a specific skill in terms of a population of examinees (e.g., Harris & Pearlman, 1978;Marks & Noll, 1967). Macready and Dayton (1977) illustrated how this point of view can be used, among other things, to determine the number of items to be used when making a mastery/nonmastery decision concerning a particular skill.…”

Section: Solutions In Terms Of Proportion Of Skills Acquiredmentioning

confidence: 99%

Determining the Length of a Criterion-Referenced Test

Wilcox

1980

Applied Psychological Measurement

View full text Add to dashboard Cite

When determining how many items to include on a criterion-referenced test, practitioners must resolve various nonstatistical issues before a particular solution can be applied. A fundamental problem is deciding which of three true scores should be used. The first is based on the probability that an examinee is correct on a "typical" test item. The second is the probability of having acquired a typical skill among a domain of skills, and the third is based on latent trait models. Once a particular true score is settled upon, there are several perspectives that might be used to determine test length. The paper reviews and critiques these solutions. Some new results are described that apply when latent structure models are used to estimate an examinee's true score. When trying to determine how many items to include on a criterion-referenced test, perhaps the most fundamental problem is that there are at least three conceptualizations, or models, of an achievement test that might be used. Each of these conceptualizations is based on a different type of true score. The first deals with the number of items an examinee would get correct if he/she were to respond to every item in some item domain. The second is concerned with the proportion of skills among a domain of skills that an examinee has acquired. Because of errors at the item level, such as guessing, this conceptualization is different from the first. The final approach is based upon latent trait models. In some cases, one model might yield substantially different results from another in terms of test length, and so the choice of a model can be crucial. Once one of the above conceptualizations is settled upon, a variety of other issues must be resolved. For example, when comparing an examinee's true score to a standard, should it be assumed that the standard is known, or should the process by which it was determined be taken into account? Should the test length problem be formulated in terms of a single examinee, a &dquo;typical&dquo; examinee, or both? How certain do we want to be of making a correct decision (classification) of an examinee? Are we willing to use a Bayesian solution? This paper has several goals. The first is to give a brief review and critique of the three general approaches that might be used when determining the length of a criterion-referenced test. The second is to describe new results on test length, and the third is to indicate possible directions for future research.

show abstract

Procedures and Criteria for Evaluating Reading and Listening Comprehension Tests

Cited by 16 publications

References 1 publication

Learning From Verbal Discourse in Educational Media: A Review of the Literature

Learning From Verbal Discourse in Educational Media: A Review of the Literature

An Approach to Measuring the Achievement or Proficiency of an Examinee

Determining the Length of a Criterion-Referenced Test

Contact Info

Product

Resources

About