An introduction to standard setting methods in dentistry

Puryer, James S; O’Sullivan, Dominic

doi:10.1038/sj.bdj.2015.755

Cited by 10 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the modified‐Angoff standard with 54.6% as a passing score will result in an 80% pass rate. This finding was similar to the cutoff scores obtained using the Angoff method by a previous study in dentistry by Puryer and O'Sullivan, 12 and passing rate trends are similar to those reported by Iweka and Tei‐Firstman 34…”

Section: Discussionsupporting

confidence: 90%

“…In this study, the judges consisted of the dean, deputy deans, professor, associate professor, and lecturer representatives from the various specialty departments. The selection is in agreement with the criteria that judges are knowledgeable about the curriculum, the abilities of the cohort, and be a balanced mix of age, gender, educational experience, and subject experience 12 . From the evidence, it was found that negligible differences in cutoff scores were established by panelists with different specializations 24,25 .…”

Section: Discussionsupporting

confidence: 52%

See 1 more Smart Citation

A comparison of different standard‐setting methods for professional qualifying dental examination

Rahman

Baharuddin

Hassan

et al. 2021

Journal of Dental Education

View full text Add to dashboard Cite

Background The outcome of assessments is determined by the standard‐setting method used. Standard setting is the process of deciding what is good enough. A cutoff score of 50% was commonly used in dental schools in Malaysia. This study aims to compare the conventional, norm‐referenced, and modified‐Angoff standard‐setting methods. Methods The norm‐referenced method of standard setting was applied to the real scores of 40 final‐year dental students on a multiple‐choice question (MCQ), a short answer question (SAQ), and an objective structured clinical examination (OSCE). A panel of 10 judges set the standard using the modified‐Angoff method for the same paper in one sitting. One judge set the passing score of 10 OSCE questions after 2 weeks. A comparison of the grades and pass/fail rates derived from the absolute standard, norm‐referenced, and modified‐Angoff methods was made. The intra‐rater and inter‐rater reliabilities of the modified‐Angoff method were assessed. Results The passing rate for the absolute standard was 100% (40/40), for the norm‐referenced method it was 62.5% (25/40), and for the modified‐Angoff method it was 80% (32/40). The modified‐Angoff method had good inter‐rater reliability of 0.876 and excellent test–retest reliability of 0.941. Conclusion There were significant differences in the outcomes of these three standard‐setting methods, as shown by the difference in the proportion of candidates who passed and failed the assessment. The modified‐Angoff method was found to have good reliability for use with a professional qualifying dental examination.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Discussionsupporting

confidence: 52%

A comparison of different standard‐setting methods for professional qualifying dental examination

Rahman

Baharuddin

Hassan

et al. 2021

Journal of Dental Education

View full text Add to dashboard Cite

show abstract

“…To aid discussion during the workshop, a reading list of relevant references was distributed to all registered participants. The references were categorised under the following subject headings: Selection of appropriate assessment tools; Selection of appropriate standard setting methods; Workplace‐based assessment; How best to assess “soft‐skills” such as professionalism, leadership and management …”

Section: Pre‐workhop Online Registrant Survey Results and Reading Listmentioning

confidence: 99%

Assessment in a global context: An international perspective on dental education

Patel

Tonni

Gadbury‐Amyot

et al. 2018

Eur J Dental Education

View full text Add to dashboard Cite

Assessments are widely used in dental education to record the academic progress of students and ultimately determine whether they are ready to begin independent dental practice. Whilst some would consider this a "rite-of-passage" of learning, the concept of assessments in education is being challenged to allow the evolution of "assessment for learning." This serves as an economical use of learning resources whilst allowing our learners to prove their knowledge and skills and demonstrating competence. The Association for Dental Education in Europe and the American Dental Education Association held a joint international meeting in London in May 2017 allowing experts in dental education to come together for the purposes of Shaping the Future of Dental Education. Assessment in a Global Context was one topic in which international leaders could discuss different methods of assessment, identifying the positives, the pitfalls and critiquing the method of implementation to determine the optimum assessment for a learner studying to be a healthcare professional. A post-workshop survey identified that educators were thinking differently about assessment, instead of working as individuals providing isolated assessments; the general consensus was that a longitudinally orientated systematic and programmatic approach to assessment provide greater reliability and improved the ability to demonstrate learning.

show abstract

“…12 Throughout the literature, the advantage of using the Yes/No method is characterised as practicability, which means it is simpler to implement and makes decision-making process easier for panel members. 8,13 The Percentage method is the most commonly used in both dental and medical education (eg Schoonheim-Klein et al 14 ), but the disadvantage of the method is known to be the "considerable variability" amongst judges because of the continuum of item probabilities. 15 Whilst the direct comparison of these two methods is not found in the context of dental or medical education to our knowledge, two comparison studies are available in the broader field of educational measurement.…”

Section: Introductionmentioning

confidence: 99%

How to improve reliability of cut‐off scores in dental competency exam: A comparison of rating methods in standard setting

Kim

Yang

2020

Eur J Dental Education

View full text Add to dashboard Cite

Aim: The aim of this study was to find ways to improve reliability of cutoff scores that are typically used to make high-stake decisions in dental education by empirically comparing two different rating methods, Yes/No and Percentage methods. Material and Methods: The two rating methods are commonly used when the Angoff's method is applied to determine a cutoff score that divides the examinees into minimally competent group (pass) and incompetent group (fail). The expert panel data were collected using both methods from 11 to 13 panel members in two consecutive years, respectively; The data were analysed within the generalisability theory framework to quantify relative influences of each factor (eg panel, item, rating rounds) on the variability of cutoff scores, standard error of measurement and panel agreement. Results: The results suggest that (a) the two methods can make a substantial difference in overall success rates for college senior students, (b) item-related variance components are generally large and whilst rater-related variance components are small, (c) standard errors of measurement for the cutoff scores decreased from Cohort 1 to Cohort 2 as the number of items are increased and as the expert panel members are more trained and (d) the Percentage method yielded higher agreement amongst the panel in both years. The results provide practical guidelines for dental educators who make efforts to control the quality of final competency exams and cutoff scores with respect to standard setting practices and panel data analysis. Conclusion: It can be concluded that evaluations with Percentage method results in more reliable outcomes compared to those with Yes/No method when criterion-referenced assessment is applied to determine the cutoff scores of competency tests at schools.

show abstract

An introduction to standard setting methods in dentistry

Cited by 10 publications

References 12 publications

A comparison of different standard‐setting methods for professional qualifying dental examination

A comparison of different standard‐setting methods for professional qualifying dental examination

Assessment in a global context: An international perspective on dental education

How to improve reliability of cut‐off scores in dental competency exam: A comparison of rating methods in standard setting

Contact Info

Product

Resources

About