Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation

Quaigrain, Kennedy; Arhin, Ato Kwamina

doi:10.1080/2331186x.2017.1301013

Cited by 117 publications

(151 citation statements)

References 13 publications

Supporting

Mentioning

104

Contrasting

Unclassified

Order By: Relevance

“…It is important to include more questions with average difficulty even though the mean difficulty level (0.56) is acceptable. A similar finding was reported in many other literatures (1, 6 and 9).…”

Section: Discussionsupporting

confidence: 92%

See 1 more Smart Citation

Post exam item analysis: Implication for intervention

Knn

2019

Preprint

View full text Add to dashboard Cite

The difficulty index, discrimination and distracter efficiency of college level exam 10 paper was analyzed as an input for taking actions in future test developments. The exam 11 papers of 176 first-year regular pre-service diploma students at Gondar CTE were analyzed 12 using descriptive analysis. Difficulty indices and distracter efficiencies were calculated using 13 Microsoft Excel 2007. Other test statistics such as mean, bi-serial correlations and reliability 14 coefficients were computed using SPSS version 20. Results showed that the mean test score, 15 out of 31, was 17.23 ± 3.85. Average difficulty and discrimination indices were 0.56 (SD 0.20) 16 and 0.16 (SD 0.28), respectively. The mean distracter efficiency was 92.1% (SD 17.2%). The 17 reliability of the test was 0.58. Out of 31, 13 (41.9%) items were either too easy or too difficult. 18 Only two items fell into good or excellent discrimination power. Inconsistency in option19 formats and inappropriate stems were observed in the exam paper. Based on the results the 20 college level exam paper has acceptable level of difficulty index and distracter efficiency. 21 (0.16, acceptable ≥ 0.4). The 22 internal consistency reliability was also less than the acceptable level (0.58, acceptable ≥ 0.7). 23 However, the average discrimination power of exam was very low Thus, future test development interventions should give due emphasis on item reliability,24 discrimination coefficients and item face validity. 25 26 3 32Higher education institutes need to combine different approaches and instruments for assessing 33 students (5). This is because students' assessment and evaluation are an integral part of the 34 teaching -learning process (2). The assessments should be relevant while tracking each student's 35 performance in a given course. Considering this, instructors at higher institutions must be aware 36 of the quality and reliability of tests. Otherwise, the final results may be influenced by the test 37 itself, which could lead to a biased assessment (5). Usually instructors receive little or no training 38 on assessment quality. If training given, it doesn't focus on strategies to construct test or item-39 writing rules but only on large-scale test administration and standardized test score interpretation 40 (2). Tavakol and Dennick (2011) pointed out the importance of post-exam item analysis to 41 improve the quality and reliability of assessments. 42 43Item-analysis is the process of collecting, summarizing and using information from students' 44 responses to assess the quality of test items (21). It allows teachers to identify: too difficult or too 45 easy items, items that do not discriminate high and low able students or items that have 46 implausible distracters (2, 3). In these cases, teachers can remove too easy or too difficult items 47

show abstract

Section: Discussionsupporting

confidence: 92%

“…This revealed that there is some gap in the preparation of good questions. A similar finding was reported in many other literatures (1, 6 and 9).…”

Section: Discussionsupporting

confidence: 92%

Post exam item analysis: Implication for intervention

Knn

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Ketika semua ini dilakukan, reliabilitas tes akan meningkat. [23]. Danuwijaya (2018) dalam hasil penelitiannya menyebutkan bahwa sebagian besar butir soal ratarata masuk kategori sukar.…”

Section: Pendahuluanunclassified

Analisis Kualitas Butir Soal Tipe Hots Pada Kompetensi Sistem Rem Di Sekolah Menengah Kejuruan

Warju

Ariyanto

Soeryanto

et al. 2020

j. pendidik. teknologi. kejuruan.

View full text Add to dashboard Cite

Tujuan dari penelitian ini adalah untuk mengetahui kualitas dari soal tipe HOTS pada kompetensi sistem rem melalui pelaksanaan analisis butir soal. Penelitian ini merupakan penelitian evaluatif. Sampel penelitian ini adalah siswa kelas XI TKRO 1 di SMKN 1 Labang Bangkalan sejumlah 32 siswa. Instrumen penelitian ini meliputi soal pilihan ganda tipe HOTS, kunci jawaban, dan lembar jawaban hasil pekerjaan siswa, yang kemudian dianalisis validitas, reliabilitas tingkat kesukaran, daya pembeda, dan efektivitas pengecoh menggunakan bantuan software Anates V4. Hasil penelitian ini menunjukkan bahwa dari total 50 soal pilihan ganda terdapat 80%soal yang masuk dalam kategori valid, sedangkan 20% soal masuk pada kategori tidak valid. Kemudian untuk reliabilitas mendapatkan nilai koefisien sebesar 0,917. Sementara itu soal yang masuk kategori mudah, sedang, dan sukar pada indeks tingkat kesukaran masing-masing sebesar 38%, 46%, dan 16%. Kemudian soal yang masuk kategori baik sekali, baik, cukup, buruk, dan tidak baik pada indeks daya beda masing-masing sebesar 10%, 42%, 28%, 6%, dan 14%. Selanjutnya pada indeks pengecoh soal yang masuk kriteria sangat baik, baik, kurang baik, buruk, dan sangat buruk masing-masing sebesar 28%, 29%, 19%, 17%, dan 7%. Dengan demikian dapat disimpulkan bahwa soal tipe HOTS pada kompetensi sistem rem layak digunakan sebagai alat penilai kompetensi siswa

show abstract

“…Ten is used to measure the extent to which an item can predict the overall performance of a test (Xue Bai & Ola, 2017). The following rules of a discriminant level similar to that used by (Quaigrain & Arhin, 2017)as shown in table 11:…”

Section: Item Discriminantmentioning

confidence: 99%

Specific Open-Ended Assessment: Assessing Students' Critical Thinking Skill on Kinetic Theory of Gases

et al. 2019

View full text Add to dashboard Cite

The test of critical thinking skills in specific topics in physics is still rarely. This study aimed to develop a specific test in critical thinking skills in the kinetic theory of gases (CTKTG) and also to assess the students' critical thinking skills. This study used the 4D method (Define, Design, Develop, and Disseminate). The CTKTG test was initially tested in four sample groups: interviews with an expert review (N = 3), professional physics teachers (N = 2), and graduate school students (N = 2), students from secondary schools (N = 29). The test was modified based on the revised results in the initial test. After that, the test was given to a group of students in class XI, who were science students (N = 55). The results showed that internal consistency from the CTKTG test was α = .89 (good). The implementation strategies and tactics are the most difficult aspect of critical thinking skill with a mean of 1.37 (very low) and basic classification is easiest with a mean of 2.84 (average). So, the findings showed that the CTKTG test can be used to measure students' critical thinking skills on the topic of the kinetic theory of gases.

show abstract

Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation

Cited by 117 publications

References 13 publications

Post exam item analysis: Implication for intervention

Post exam item analysis: Implication for intervention

Analisis Kualitas Butir Soal Tipe Hots Pada Kompetensi Sistem Rem Di Sekolah Menengah Kejuruan

Specific Open-Ended Assessment: Assessing Students' Critical Thinking Skill on Kinetic Theory of Gases

Contact Info

Product

Resources

About