Multiple Processing Strategies and the Construct Validity of Verbal Reasoning Tests

Abstract

This study examines the influence of processing strategies, and the associated metacomponents that determine when to apply them, on the construct validity of a verbal reasoning test. Three strategies for solving verbal analogy items were examined: a rule-oriented strategy, an association strategy, and a partial rule strategy. Construct validity was studied in two separate stages: construct representation and nomothetic span. For construct representation, evidence was obtained that all three strategies, and their related metacomponents, are associated with performance on analogy items. For nomothetic span, the current study found that all three strategies contribute to individual differences in verbal reasoning and to the predictive validity of the test. The results of this study also point to the utility of metacomponents as constructs for describing and understanding test performance. Implications of the results for test development and theories of aptitude are elaborated.

Generating items during testing: Psychometric issues and models

“…First, item properties are often predicted at least moderately well. Studies from my laboratory have concerned ten different item types, including verbal analogies and verbal classifications (Embretson, 1984;Embretson, 1985;Embretson & Schneider, 1989;Embretson, Schneider & Roth, 1985;Whitely, 19801), geometric analogies and geometric classifications (Whitely & Schneider, 1981), series completion (Embretson, 1985), paragraph comprehension (Embretson & Wetzel, 1987), spatial folding (Embretson, 1991(Embretson, , 1994, mathematics word problems (Embretson, 1995a), matrix completion problems (Embretson, 1995c; and, most recently, spatial construction items. Prediction levels comparable to a multiple correlation of at least .70 are usually obtained.…”

Controlling for Background Beliefs When Developing Multiple‐Choice Critical Thinking Tests

“…Nor can the criticism be countered by pointing to the large amount of correlation data Norris Background Beliefs and Critical Thinking Testing -5 relating performance on the Watson-Glaser test to other variables (Watson & Glaser, 1980b). This data provides evidence on the convergent and discriminant validity of the test, but it is not compelling when the same data used to elaborate the nomothetic span of a test is also used to clarify the construct the test measures in the first place (Embretson, Schneider, & Roth, 1986). This is especially true for the construct of critical thinking.…”

A risk factor model predicting the relationship between stress and performance in Explosive Ordnance Disposal (EOD) training

“…The second limitation is that measures of general cognitive ability focus on the outcome of performance and overlook important process elements of performance, such as problem-solving approaches (Embretson, Schneider, & Roth, 1986;Sigel, 1963). hi support of the difference between general cognitive ability and problem solving processes, several studies have found that measures of general intelligence tend to have low correlations with real-life problem-solving performance (Epstein & Meier, 1989;Heppner & et al, 1982;Spivak, Shure, & Platt, 1985;Stemberg, Wagner, Williams, & Horvath, 1995).…”