A Generalizability Approach to the Measurement of Score Reliability of the Teacher Assessment Literacy Questionnaire

Alkharusi, Hussain

doi:10.5296/jse.v2i2.1495

Cited by 4 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The interview findings suggest that teachers' assessment competence is perceived to be modest, with low levels of assessment literacy and practice, and it noted that this assessment literacy impacts students' motivation to learn, and engage, and affects their overall competence. This finding is consistent the results of other studies (Al-Bahlani,2017;Alkharusi, 2011b; 2011c; Alkharusi et al, 2011, Alkharusi et al, 2012Lyon, 2011;Mertler and Campbell, 2005). The study also found that there The overall assessment literacy was deemed low with an average score of 8.81/35 (25%) compared to the overall teachers' perceived practice (competence total mean score 3.11 (moderately competent), practice total mean score 1.66).…”

Section: Discussionsupporting

confidence: 90%

“…In the last research question interview findings supported the quantitative data, indicating that teachers' assessment competence is perceived as modest, impacting students' motivation, engagement, and overall competence. This aligns with previous studies (Al-Bahlani, 2017; Alkharusi, 2011b, 2011c; Alkharusi et al, 2011Alkharusi et al, , 2012Lyon, 2011;Mertler & Campbell, 2005). The study recommended a multifaceted approach, incorporating pre-service, in-service, and continuous professional development to enhance teachers' assessment competence.…”

Section: Discussionsupporting

confidence: 86%

See 1 more Smart Citation

Teachers’ Competence in The Educational Assessment of Students in A Competence Based Education in Amhara Region, Ethiopia: The Case of TVET Teachers and Assessors

Asres,

Abebe

2024

Preprint

View full text Add to dashboard Cite

This study investigates the assessment literacy, perceived practice, and observed practice of CBA practices among TVET teachers and assessors in private TVET colleges in South Gonder Zone, Ethiopia. The participants included 49 teachers and 14 assessors. Data collection utilized a modified four-option multiple-choice test, the Teacher Assessment Competency Inventory Questionnaire, a five-point Likert scale for perceived and observed practices, a semi-structured observation checklist, and interviews. Descriptive analysis, including mean scores, standard deviation, one-sample t-test, and proportions (%), was employed to analyze the data. Pearson correlation test was used to explore relationships between teacher assessment test scores, observed practices, and perceived practices. Findings revealed that private TVET teachers in the Amhara National Regional State lacked competence in assessing students' competence, falling short of overall and specific standards in both the assessment competence test and real classroom practice. Despite this, teachers modestly perceived their assessment practices across competency domains. Consequently, there were significant differences between teachers' actual competence and practice in educational assessments and their self-perceived practice. Qualitative approaches, employing theme analysis of assessment practice from semi-structured observations and interviews, highlighted poor performance among teachers on most assessment criteria. A small percentage demonstrated competence in specific concepts. Overall, the majority of teachers exhibited low levels of assessment literacy, suggesting a need for professional support in educational assessment approaches to enhance competence among TVET teachers.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Discussionsupporting

confidence: 86%

Teachers’ Competence in The Educational Assessment of Students in A Competence Based Education in Amhara Region, Ethiopia: The Case of TVET Teachers and Assessors

Asres,

Abebe

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…This has made assessments important in learning, and would require for lecturers to undertake appropriate assessments on student learning (Stiggins et al, 1989;Popham, 2004a, b;Carey, 1994). The learning outcomes determined by the lecturers are great determinants of the students' destiny as far as their education is concerned (Ames, 1992;Harlen and Crick, 2003;Alkharusi, 2008). The differences discovered in lecturers' assessment competencies and practices in the specialisation of education with other specialisations were expected.…”

Section: Discussionmentioning

confidence: 98%

Self-perceived assessment competencies and practices among university lecturers

Matovu

Zubairi

2014

Journal of Applied Research in Higher Education

View full text Add to dashboard Cite

Purpose – The purpose of this paper is to analyse the lecturers’ self-perceived competencies and practices in assessing students. Design/methodology/approach – An Assessment Practices Inventory Modified was administered to a sample of 329 randomly selected lecturers from six universities in Uganda. Factor analysis and multivariate analysis of variance (MANOVA) were used to address the research questions. Findings – The results of factor analysis yielded a factor structure of four variables; design, administration, interpretation, and application. The MANOVA multivariate test results highlighted differences in assessment competencies and practices among lecturers in the different academic levels (Wilks’ λ=0.732, F(16, 313)=5.624, p<0.05, η2=0.075), and in the interaction between type of university, specialisations, and academic levels (Roy's largest root=0.073, F(8, 313)=2.543, p<0.05, η2=0.068). The Tukey HSD post hoc test results revealed that lecturers in the specialisation of education were different from their counterparts in other specialisations, in interpreting assessment results. Academic levels differences existed in all the dependent variables (design, administration, interpretation, and application). No differences existed in assessment competencies and practices between lecturers in the different types of universities. Originality/value – Lecturers have been found to differ in their assessment competencies and practices, according to their specialisations and academic levels. It has been recommended in this study, that, assessment training programmes be made mandatory to all lecturers in universities, in order to bridge the gap their competencies and practices in assessing students.

show abstract

The affectability of writing assessment scores: a G-theory analysis of rater, task, and scoring method contribution

Khodi

2021

Lang Test Asia

View full text Add to dashboard Cite

The present study attempted to to investigate factors which affect EFL writing scores through using generalizability theory (G-theory). To this purpose, one hundred and twenty students participated in one independent and one integrated writing tasks. Proceeding, their performances were scored by six raters: one self-rating, three peers,-rating and two instructors-rating. The main purpose of the sudy was to determine the relative and absolute contributions of different facets such as student, rater, task, method of scoring, and background of education to the validity of writing assessment scores. The results indicated three major sources of variance: (a) the student by task by method of scoring (nested in background of education) interaction (STM:B) with 31.8% contribution to the total variance, (b) the student by rater by task by method of scoring (nested in background of education) interaction (SRTM:B) with 26.5% of contribution to the total variance, and (c) the student by rater by method of scoring (nested in background of education) interaction (SRM:B) with 17.6% of the contribution. With regard to the G-coefficients in G-study (relative G-coefficient ≥ 0.86), it was also found that the result of the assessment was highly valid and reliable. The sources of error variance were detected as the student by rater (nested in background of education) (SR:B) and rater by background of education with 99.2% and 0.8% contribution to the error variance, respectively. Additionally, ten separate G-studies were conducted to investigate the contribution of different facets across rater, task, and methods of scoring as differentiation facet. These studies suggested that peer rating, analytical scoring method, and integrated writing tasks were the most reliable and generalizable designs of the writing assessments. Finally, five decision-making studies (D-studies) in optimization level were conducted and it was indicated that at least four raters (with G-coefficient = 0.80) are necessary for a valid and reliable assessment. Based on these results, to achieve the greatest gain in generalizability, teachers should have their students take two writing assessments and their performance should be rated on at least two scoring methods by at least four raters.

show abstract

A Generalizability Approach to the Measurement of Score Reliability of the Teacher Assessment Literacy Questionnaire

Cited by 4 publications

References 10 publications

Teachers’ Competence in The Educational Assessment of Students in A Competence Based Education in Amhara Region, Ethiopia: The Case of TVET Teachers and Assessors

Teachers’ Competence in The Educational Assessment of Students in A Competence Based Education in Amhara Region, Ethiopia: The Case of TVET Teachers and Assessors

Self-perceived assessment competencies and practices among university lecturers

The affectability of writing assessment scores: a G-theory analysis of rater, task, and scoring method contribution

Contact Info

Product

Resources

About