General fluid/inductive reasoning battery for a high-ability population

Abstract

The validity of studies investigating interventions to enhance fluid intelligence (Gf) depends on the adequacy of the Gf measures administered. Such studies have yielded mixed results, with a suggestion that Gf measurement issues may be partly responsible. The purpose of this study was to develop a Gf test battery comprising tests meeting the following criteria: (a) strong construct validity evidence, based on prior research; (b) reliable and sensitive to change; (c) varying in item types and content; (d) producing parallel tests, so that pretest-posttest comparisons could be made; (e) appropriate time limits; (f) unidimensional, to facilitate interpretation; and (g) appropriate in difficulty for a high-ability population, to detect change. A battery comprising letter, number, and figure series and figural matrix item types was developed and evaluated in three large-N studies (N = 3,067, 2,511, and 801, respectively). Items were generated algorithmically on the basis of proven item models from the literature, to achieve high reliability at the targeted difficulty levels. An item response theory approach was used to calibrate the items in the first two studies and to establish conditional reliability targets for the tests and the battery. On the basis of those calibrations, fixed parallel forms were assembled for the third study, using linear programming methods. Analyses showed that the tests and test battery achieved the proposed criteria. We suggest that the battery as constructed is a promising tool for measuring the effectiveness of cognitive enhancement interventions, and that its algorithmic item construction enables tailoring the battery to different difficulty targets, for even wider applications. Keywords Intelligence. Fluid ability. Gf. Working memory training. Reasoning. Item-response theory. Test assembly General fluid ability (Gf) is Bat the core of what is normally meant by intelligence^(Carroll, 1993, p. 196), and has been shown empirically to be synonymous with general cognitive ability (g), at least within groups with roughly comparable opportunities to learn (Valentin Kvist & Gustafsson, 2008). Gf has been viewed as an essential determinant of one's ability to solve a wide range of novel real-world problems (Schneider & McGrew, 2012). Perhaps because of its association with diverse outcomes, there has been a longstanding interest in improving Gf (i.e., intelligence) through general schooling

The Precipitous Decline in Reasoning and Other Key Abilities with Age and Its Implications for Federal Judges

“…There are precedents for research programs sponsored by the federal government. The U. S. Office of the National Director of Intelligence includes, among its programs, Strengthening Abstract Reasoning and Problem-solving (SMART; Kyllonen et al ( 2019 )). The goals of SMART are to develop tests for high-ability people and then try to validate interventions to improve the agents’ abstract reasoning and critical problem-solving ability.…”

Assessing Inductive Reasoning, Scientific Reasoning and Science Motivation

“…As for the combination of form of reasoning and learning Regarding materials for constructing problems, elements in IR tasks are made of words, numbers, and figures (e.g., geometry, animal, toy, and artifact). The figural object task was the most common one chosen to assess IR abilities of children (Csapó et al, 2014;Csapó et al, 2009;Hamers et al, 1998;Jeotee, 2012;Kambeyo & Wu, 2018;Kyllonen et al, 2019;Muniz et al, 2012;Roberts et al, 2000;Schroeders & Wilhelm, 2010;Stevenson et al, 2013;Wesiak, 2003). Up to 86.5 percent of the selected studies implemented the figure as a main material in assessing IR capacity.…”

“…Nearly 35% of the reviewed studies utilized the numeric materials for the main tests in their research (e.g. Bühner et al, 2008;Csapó, 1997;Csapó et al, 2009;Kambeyo & Wu, 2018;Korom et al, 2017;Kyllonen et al, 2019;Nikolov & Csapó, 2018;Wesiak, 2003). The tasks with numeric materials are not only to assess thinking skills, but they are probably also helpful to investigate the participants' mathematical competency in general.…”

“…Hence, studies used the figural and geometric objects as their main instruments (e.g. Blum et al, 2016;Kyllonen et al, 2019;Muniz et al, 2012;Schroeders & Wilhelm, 2010;Stevenson et al, 2013;Tunteler et al, 2008;Tzuriel & George, 2009;Vogelaar et al, 2019;Wang, 2008), accounting for more than 80%. Although non-verbal tasks were the most common items composed to assess IR (18.9%), verbal elements play a particular role in many learning tasks in school contexts.…”

“…In a comparison of reasoning ability in males and females, inconsistent findings were reported in different studies in both IR and SR. Particularly, some papers reported a significant difference in IR between male and female students, favouring males (Blum et al, 2016;Jeotee, 2012;Kambeyo & Wu, 2018;Kyllonen et al, 2019;Tairab, 2015;Tekkaya & Yenilmez, 2006;Venville & Oliver, 2015). In contrast, another study showed that girls got higher scores than boys on the IR test (Díaz- Morales & Escribano, 2013).…”