Ítems contextualizados de PISA Ciencias: La conexión entre las demandas cognitivas y las características de contexto de los ítems

Ruiz-Primo, María Araceli; Li, Min

doi:10.7203/relieve.22.1.8280

Cited by 3 publications

(2 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A wealth of evidence speaks to the influence of different item features on the examinees' performance on tests. For example, we know that the performance of students is instable across item formats (Ruiz-Primo et al, 1993); that construct equivalence may vary depending on the ways in which items are designed (Rodriguez, 2003); and that even small changes in wording may cause translated items to function differentially (Ercikan et al, 2014).…”

Section: Items As Samples Of Encoded Cultural Experiencementioning

confidence: 99%

See 1 more Smart Citation

The Semiotics of Test Design: Conceptual Framework on Optimal Item Features in Educational Assessment Across Cultural Groups, Countries, and Languages

Solano‐Flores

2021

Front. Educ.

View full text Add to dashboard Cite

This paper offers a conceptual framework on test design from the perspective of social semiotics. Items are defined as arrangements of features intended to represent information, convey meaning, and capture information on the examinees’ knowledge or skills on a given content. The conceptual framework offers a typology of semiotic resources used to create items and discusses item representational complexity—the multiple ways in which the semiotic resources of an item are related to each other—and item semiotic alignment—the extent to which examinees share cultural experience encoded by items. Since the ability to make sense of items is shaped by the examinees’ level of familiarity with the social conventions underlying the ways in which information is represented, unnecessary representational complexity and limited semiotic alignment may increase extraneous item cognitive load and adversely impact the performance of examinees from certain populations. Semiotic test design allows specification of optimal pools of semiotic resources to be used in creating items with the intent to minimize representational complexity and maximize semiotic alignment for the maximum number of individuals in diverse populations of examinees. These pools of semiotic resources need to be specific to the content assessed, the characteristics of the populations of examinees, the languages involved, etc., and determined based on information produced by cross-cultural frequency analyses, cognitive interviews, focus groups, and expert panels.

show abstract

Section: Items As Samples Of Encoded Cultural Experiencementioning

confidence: 99%

“…Contexts are very common in current large-scale assessment programs. For example, a study on the use of contexts in PISA 2006 and PISA 2009 items found that about one third of the sample of items examined contained contexts in the form of a narrative (Ruiz-Primo and Li, 2016).…”

Section: Contextsmentioning

confidence: 99%

The Semiotics of Test Design: Conceptual Framework on Optimal Item Features in Educational Assessment Across Cultural Groups, Countries, and Languages

Solano‐Flores

2021

Front. Educ.

View full text Add to dashboard Cite

show abstract

Language in science performance: do good readers perform better?

2019

View full text Add to dashboard Cite

Science performance is highly affected by students' reading comprehension. Recently, there has been a growing attention to the role of linguistic features for science performance, but findings are ambivalent, especially when looking into item word count. The aim of this study was to investigate the interaction of students' reading comprehension and item word count of given science measures on performance, controlling for students' cognitive abilities, gender, family language, and school track. The sample consisted of N = 2051 German students in grades 10 and 11. Students completed (scientific) literacy measures. We then applied a multilevel logistic regression to investigate the hypothesized interaction effect of reading comprehension and word count on students' science performance. The results showed a significant interaction of students' reading comprehension and word count on science performance, controlling for several covariates. Particularly students with high reading comprehension benefit from science items with increasing word count. Our findings empirically support previous research, showing that reading comprehension is crucial for science performance and enhances the interaction between reading comprehension and linguistic features of written text in science subjects. Finally, theoretical and practical implications and limitations of this study are discussed.

show abstract

Diseño de un Instrumento para Evaluar el Valor Social Subjetivo de la Educación en Estudiantes, Docentes y Familias: Resultados de un Ensayo Piloto / Design of an Instrument to Assess the Subjective Social Value of Education in Students, Teachers and Families: Results of a Pilot Study

Sancho-Álvarez¹,

Meliá²,

Such³

2017

riee

View full text Add to dashboard Cite

Encontramos multitud de estudios sobre el diseño y validación de instrumentos de evaluación sobre variables socio-afectivas complejas. En estos casos ha sido evidente que un desarrollo sistemático de estandarización ha mejorado las propiedades métricas y su adecuación. Por ello planteamos algunos resultados previos sobre un estudio piloto para la construcción de un instrumento que evalúe el Valor Social Subjetivo de la Educación en tres audiencias (estudiantes, profesorado y familias). En este caso se presentan las propiedades métricas encontradas y las diferencias entre grupos con 131 estudiantes, 28 docentes y 36 familiares. En general los datos son adecuados en las tres escalas implementadas, sin embargo se encuentran nuevas líneas de indagación. Como conclusión se reflexiona sobre la necesidad de asegurar la fiabilidad y la validez de la escala para poder mejorar la investigación.Palabras clave: Evaluación educativa, Indicadores educativos, Metodología de investigación, Medición, Estudiantes, Docentes, Familias.We found many studies about the design and validation of evaluation instruments with complex socio-affective variables. In these cases, it has been clear that a systematic scheme of standardization has improved the metric properties and their suitability. Therefore, we propose some preliminary results of a pilot study for the construction of an instrument to assess the Subjective Social Value of Education in three audiences (students, teachers and families). In this case, are presented the metric properties found and the differences between groups with 131 students, 28 teachers and 36 family members. In general, in the in the three scales implemented the dates are suitable, however we found new lines of inquiry. In conclusion, we reflect about the need to ensure the reliability and the validity of test for we can to improve the study.

show abstract

Ítems contextualizados de PISA Ciencias: La conexión entre las demandas cognitivas y las características de contexto de los ítems

Cited by 3 publications

References 1 publication

The Semiotics of Test Design: Conceptual Framework on Optimal Item Features in Educational Assessment Across Cultural Groups, Countries, and Languages

The Semiotics of Test Design: Conceptual Framework on Optimal Item Features in Educational Assessment Across Cultural Groups, Countries, and Languages

Language in science performance: do good readers perform better?

Diseño de un Instrumento para Evaluar el Valor Social Subjetivo de la Educación en Estudiantes, Docentes y Familias: Resultados de un Ensayo Piloto / Design of an Instrument to Assess the Subjective Social Value of Education in Students, Teachers and Families: Results of a Pilot Study

Contact Info

Product

Resources

About