Rasch-Based Objective Standard Setting for University Placement Test

Bichi, Ado Abdu; Talib, Rohaya; Embong, Rahimah; Mohamed, Hasnah Binti; Ismail, Mohd Sani; Ibrahim, Abdallah

doi:10.14689/ejer.2019.84.3

Cited by 5 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Rasch model can be used to analyze dichotomous and polytomous data (Alias et al, 2019;Andrich, 1978). Furthermore, Bichi et al (2019) explained that the interpretation via Rasch analysis provides thorough information about the item structure, thus, can be used to validate students' ability and appropriateness of the items. Scholars explain that Rasch model analysis may offer a robust assessment for reliability and validity with a small sample as well as provide an elaboration of the respondents and item analysis (Che Lah et al, 2022Lah et al, , 2023.…”

Section: The Rasch Measurement Modelmentioning

confidence: 99%

Psychometric Properties Assessment of Postgraduate Self-Determined Learning Questionnaire (PSLQ): A Mixed Method Approach

Chia Chun,

Samsudin,

Abdullah

2024

Sage Open

View full text Add to dashboard Cite

Postgraduate students need to be equipped with self-determined learning skills in order to meet the demands of higher degree and employment in the fourth industrial revolution (IR 4.0) environment. However, the lack of analysis conducted on item level for validating the measurement of postgraduate students’ self-determined learning skills prompted the development of The Postgraduate Self-determined Learning Questionnaire (PSLQ) to fill in the research gaps. This paper aimed to assess the psychometric properties of the newly developed 42-item instrument through a mixed method approach as no study has evaluated the psychometric properties of PSLQ at the item level through Rasch model analysis. Quantitatively, 440 postgraduate students were sampled from a higher education institution in Malaysia to answer the questionnaire. Qualitatively, 10 respondents from science and non-science disciplines were interviewed. The paper reports findings on the psychometric properties of PSLQ through Rasch Rating Analysis and participants’ perspectives on the item validity. The results revealed that the items in PSLQ demonstrated good psychometric properties for the measurement of self-determined learning skills of postgraduate students. Implications of the study were discussed in this paper. Future studies can further investigate the psychometric properties of PSLQ according to students’ discipline of study.

show abstract

Section: The Rasch Measurement Modelmentioning

confidence: 99%

Psychometric Properties Assessment of Postgraduate Self-Determined Learning Questionnaire (PSLQ): A Mixed Method Approach

Chia Chun,

Samsudin,

Abdullah

2024

Sage Open

View full text Add to dashboard Cite

show abstract

“…According to Glynn [19], the Rasch model has been widely applied in educational measurement including the Programme for International Student Assessment (PISA), Trends in International Mathematics and Science Study (TIMSS) and other comparative tests. The Rasch model was also used in different countries and universities to validate an entrance test such as the validation of the university entrance English test of Vietnam National University [20], the validation of the university placement test in Nigeria [21] and the validation of the Chemistry national exam in Indonesia [22].…”

Section: Test Validation Using the Rasch Modelmentioning

confidence: 99%

Validation of the teacher education institution’s entrance test using the Rasch model

Malonisio,

Malonisio

2023

ijirss

View full text Add to dashboard Cite

This study aimed to evaluate the reliability and validity of the College Entrance Test (CET) used by a teacher education institution to assess the general and specialization knowledge of prospective students in English, Mathematics, Science and Social Studies. The Rasch model was employed to analyze the data collected from the sample of 250 test takers for the general knowledge test, 122 for specialization in English, 74 for Mathematics, 122 for science and 77 for Social Studies. The measurement analysis components including person and item reliability, unidimensionality, person-item map, fit statistics, Point Measure Correlation (PTMEA) and item local dependence were used. The study findings indicated that some test components had poor person reliability and the degree of item difficulty was higher than the students' abilities. There were also concerns about the conformity to the unidimensionality criteria, suggesting an analysis of items that might form another dimension in the constructs of the tests, although each of the items is independent. Furthermore, some items were misfitting or overfitting the Rasch model. In conclusion, the CET needs improvement to ensure its quality as a reliable and valid selection tool for the college. The study's results provide significant insights into the CET's strengths and weaknesses that can guide the test developers in revising and enhancing the CET to effectively measure the general and specialization knowledge of test takers in English, Mathematics, Science and Social Studies.

show abstract

“…Expert judgments are also used in this method, but the goal is not to specify the ratio/number of the correct responses; instead, experts discuss the essential content that might indicate the test takers' achievement (Sondergeld et al, 2020). As one of the modern criterion-based standard setting methods (Bichi et al, 2019), OSS method based on Rasch measurement model and Wright and Grosse's (1993) standard setting principles considers the expert opinions, test takers' performance and test/item difficulty at the same time (Khatimin et al, 2013). Through Rasch model, the measurement outputs are displayed on the logit scale, and the raw score can be analyzed on this scale in regard to the task/content achievement (Sondergeld et al, 2020).…”

Section: Review Of Literaturementioning

confidence: 99%

Standard Setting in Academic Writing Assessment through Objective Standard Setting Method

Fişne¹,

Şata

Karakaya³

2022

International Journal of Assessment Tools in Education

View full text Add to dashboard Cite

Performance standards have important consequences for all the stakeholders in the assessment of L2 academic writing. These standards not only describe the level of writing performance but also provide a basis for making evaluative decisions on the academic writing. Such a high-stakes role of the performance standards requires the enhancement of objectivity in standard setting procedure. Accordingly, this study aims to shed light upon the usefulness of Objective Standard Setting (OSS) method in specifying the levels of proficiency in L2 academic writing. On the basis of the descriptive research design, the sample of this research includes the examinees and raters who were student teachers at the university level. Essay task and analytical writing scoring rubric were employed as the data collection tools. In data analysis, OSS method and two-step cluster analysis were used. The analysis results of OSS method based on many-facet Rasch measurement model (MFRM) outline the distribution of the criteria into the levels of proficiency. Also, the main findings in OSS method were validated with twostep cluster analysis. That is, OSS method may be practically used to help the stakeholders make objective judgments on the examinees' target performance.

show abstract

Rasch-Based Objective Standard Setting for University Placement Test

Cited by 5 publications

References 8 publications

Psychometric Properties Assessment of Postgraduate Self-Determined Learning Questionnaire (PSLQ): A Mixed Method Approach

Psychometric Properties Assessment of Postgraduate Self-Determined Learning Questionnaire (PSLQ): A Mixed Method Approach

Validation of the teacher education institution’s entrance test using the Rasch model

Standard Setting in Academic Writing Assessment through Objective Standard Setting Method

Contact Info

Product

Resources

About