Modeling the interaction between resilience and ability in assessments with allowances for multiple attempts

Zhang, Susu; Bergner, Yoav; DiTrapani, Jack; Jeon, Minjeong

doi:10.1016/j.chb.2021.106847

Cited by 8 publications

(2 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the known difficulty information of the exam items as labels, a machine learning model is constructed to predict item difficulty. Traditional machine learning models include regression analysis [3], support vector machines (SVMs) [34], decision trees [35,36], Random Forests [37], and shallow BP neural networks [38]. On the other hand, deep learning models such as Convolutional Neural Networks (CNNs) [39], Recurrent Neural Networks (RNNs) [40] and Long Short-Term Memory (LSTM) [41] neural networks are also used.…”

Section: A Summary and Classification Of Item Difficulty Estimation M...mentioning

confidence: 99%

Novel Feature-Based Difficulty Prediction Method for Mathematics Items Using XGBoost-Based SHAP Model

Yi,

Sun,

2024

Mathematics

View full text Add to dashboard Cite

The level of difficulty of mathematical test items is a critical aspect for evaluating test quality and educational outcomes. Accurately predicting item difficulty during test creation is thus significantly important for producing effective test papers. This study used more than ten years of content and score data from China’s Henan Provincial College Entrance Examination in Mathematics as an evaluation criterion for test difficulty, and all data were obtained from the Henan Provincial Department of Education. Based on the framework established by the National Center for Education Statistics (NCES) for test item assessment methodology, this paper proposes a new framework containing eight features considering the uniqueness of mathematics. Next, this paper proposes an XGBoost-based SHAP model for analyzing the difficulty of mathematics tests. By coupling the XGBoost method with the SHAP method, the model not only evaluates the difficulty of mathematics tests but also analyzes the contribution of specific features to item difficulty, thereby increasing transparency and mitigating the “black box” nature of machine learning models. The model has a high prediction accuracy of 0.99 for the training set and 0.806 for the test set. With the model, we found that parameter-level features and reasoning-level features are significant factors influencing the difficulty of subjective items in the exam. In addition, we divided senior secondary mathematics knowledge into nine units based on Chinese curriculum standards and found significant differences in the distribution of the eight features across these different knowledge units, which can help teachers place different emphasis on different units during the teaching process. In summary, our proposed approach significantly improves the accuracy of item difficulty prediction, which is crucial for intelligent educational applications such as knowledge tracking, automatic test item generation, and intelligent paper generation. These results provide tools that are better aligned with and responsive to students’ learning needs, thus effectively informing educational practice.

show abstract

Section: A Summary and Classification Of Item Difficulty Estimation M...mentioning

confidence: 99%

Novel Feature-Based Difficulty Prediction Method for Mathematics Items Using XGBoost-Based SHAP Model

Yi,

Sun,

2024

Mathematics

View full text Add to dashboard Cite

show abstract

“…This might include time spent on individual tasks or overall time spent in the assessment process. Another factor is the continuation of attempts (Zhang et al, 2021), which can indicate whether learners have abandoned the assessment process or continued, especially after giving incorrect answers. More advanced analysis methods also allow for the detection of the deviation or alignment of given answers with common mistakes (Barana et al, 2019).…”

Section: Automated Assessmentmentioning

confidence: 99%

Toward a categorisation of indicators for assessment analytics

Joana¹,

Greiff²

2023

View full text Add to dashboard Cite

Recent advancements in assessment analytics provide potential to support learning processes and provide them with relevant informative feedback when needed. Yet, few well-defined indications yield valid data points for assessment analytics. The categorization of indicators that is presented here is designed to provide insights into the possible approaches to assessment and the meaningful connection to assessment analytics. Ethics, social responsibility, privacy, and data protection must be fully respected when following the categorization of indicators for assessment analytics.

show abstract