Temporal stability of objective structured clinical exams: a longitudinal study employing item response theory

Baig, Lubna; Violato, Claudio

doi:10.1186/1472-6920-12-121

Cited by 18 publications

(12 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…By using these methods, it will be possible to determine if sources of bias lay within the exam items rather than the examiners. Consideration must be taken to ensure the assessment items accurately and fairly re ect the assessment objectives [93].…”

Section: Assessment Toolsmentioning

confidence: 99%

Ethnic and Gender Biases in Clinical Performance Assessment (CPA) in Healthcare Education: A Systematic Review

Chao¹,

Violato²,

Concannon³

et al. 2019

Preprint

View full text Add to dashboard Cite

Background: Several forms of bias, including ethnic and gender bias, are thought to impact evaluations on Clinical Performance Assessments (CPAs). Unfairness may influence student learning attitudes if a loss of trust causes a lack of engagement in learning. Understanding the biases occurring in CPAs can lead to well-designed examiner training to ensure equality and fairness. The purpose of this systematic review is to determine the current evidence in the literature for ethnic and/or gender bias by examiners evaluating pre-licensure healthcare students in CPAs using standardized patients (SPs). Methods: Literature was systematically searched in CINAHL, PubMed and Medline from inception to February 2019, and no date range was set. Studies related to the investigation of ethnic and/or gender biases occurring in CPAs using SPs for examining health professions students were selected. A systematic review was conducted to assess the methodological quality and strength of evidence of relevant research and to identify if any potential ethnic and/or gender bias occurred in CPAs. The Guidelines for Critical Review were used to appraise the selected studies. Results: Nine studies published from 2003 to 2017 were retrieved for review. Three studies met all the Guidelines for Critical Review quality criteria, indicating stronger evidence of their outcomes, two of the studies reported ethnic and/or gender bias existing in the CPAs. Overall, four studies found ethnic and/or gender bias in CPAs, but all study results had small effect sizes. Conclusions: No systematic and consistent bias was found across the studies; nonetheless, the possibility of ethnic or gender bias by some examiners cannot be ignored. To minimize potential examiner bias, the investigation of Frame of Reference training, multiple examiners per station, and combination assessments in CPAs is recommended.

show abstract

Section: Assessment Toolsmentioning

confidence: 99%

Ethnic and Gender Biases in Clinical Performance Assessment (CPA) in Healthcare Education: A Systematic Review

Chao¹,

Violato²,

Concannon³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…As a rule, in numerous textbooks and monographs, the basic issues of measuring instruments development and application in assessment and accreditation based on classical or modern test theory are discussed, methodical approaches are proposed to the evaluation of measurement results quality and their scaling (9)(10)(11)(12) The problem of primary and secondary interpretation of data is mainly addressed by the authors of articles, considering it in the context of analyzing the results of certification, certification or accreditation in a certain field of professional activity (13)(14)(15)(16)(17). In particular, a number of authors of the articles discuss in a fragmented manner the interpretation and improvement of the quality of evaluation process results in relation to the problem of improving the professional performance of health care professionals (18)(19)(20)(21)(22)(23)(24)(25).…”

Section: Analysis Of Foreign Studiesmentioning

confidence: 99%

Сross-analysis of big data in accreditation of health specialists

et al. 2018

View full text Add to dashboard Cite

Objective: The relevance of this study is due to the mass accreditation of health professionals that is developing in Russia, which requires innovative measurement tools and opens new opportunities for a well-founded cross-analysis of specialists' professional readiness quality. Purpose of the study: The purpose of this article is to present approved methodical approaches to the transformation of accreditation data into a format suitable for secondary analysis of medical schools graduates quality based on the requirements of Professional Standards. Method: The leading methods of secondary data analysis are: a) codification of indicators in the primary data accumulation array; b) statistical processing of study results (evaluation of the relationships between the arrays of primary data accumulation and instrumental data, the correlation of test scores obtained by accreditation results with the labor functions of Professional Standards); c) the creation of representative samples for data analysis. The implementation of methods is carried out in the mode of working with arrays of big data, which also uses the method of cross-analysis to identify additional factors that affect to specialists' professional readiness quality. Results: As a results of the research, there were: 1) approaches to the codification of data in the array and their secondary analysis were developed; 2) three samples were constructed with an estimation of representativeness for different strata, including subjects, assignments and corresponding labor functions; 3) the matrix of primary data in the specialty "Pediatrics" was verified using the example of the results of students from 50 medical universities in Russia. Conclusion: Approbation of methods of secondary data analysis conducted on representative samples of the subjects showed the effectiveness of the developed approaches that should be used when analyzing large data sets in the procedures of certification or accreditation. The materials of the article can be useful for specialists in the field of assessing the quality of education or assessing the professional readiness of health professionals, managers, professors and pedagogical staff of medical schools, specialists of centers for independent assessment of qualifications.

show abstract

“…The first stage includes traditional testing. The second stage is the evaluation of practical skills with the help of special simulation stations as objective structured clinical exam (OSCE) (Baig & Violato, 2012). The third stage is the assessment of the level of Sizova et al / Fairness Principle in Accreditation of Health Specialists: The Differential Item Functioning Method 2 / 8 mastering the labor functions of professional standards using situational tasks.…”

Section: Introductionmentioning

confidence: 99%

Fairness Principle in Accreditation of Health Specialists: The Differential Item Functioning Method

Sizova

Semenova

Naydenova

et al. 2019

EURASIA J MATH SCI T

View full text Add to dashboard Cite

The main purpose of this article is to present a Differential Item Functioning method of item analysis. It is designed to minimize the discriminatory effect of individual items in the accreditation of graduates of medical universities with different training programs. The one-parameter Item Response Theory model is used to align graduates' rights in accreditation. The measure of the difference in the location of the item characteristic curves constructed for different graduates' samples is presented using the difficulty estimates. For the development of the methodology, a number of research questions are posed, the solution of which made it possible to analyze the items bias for the "Polyclinic" discipline and the range of item difficulties from 1.5 to 2, 5 logits. The item bank was cleaned and an interpretation of decisions on items release and correction for the cases of different arrangement of their characteristic curves was presented.

show abstract

Temporal stability of objective structured clinical exams: a longitudinal study employing item response theory

Cited by 18 publications

References 20 publications

Ethnic and Gender Biases in Clinical Performance Assessment (CPA) in Healthcare Education: A Systematic Review

Ethnic and Gender Biases in Clinical Performance Assessment (CPA) in Healthcare Education: A Systematic Review

Сross-analysis of big data in accreditation of health specialists

Fairness Principle in Accreditation of Health Specialists: The Differential Item Functioning Method

Contact Info

Product

Resources

About