Psychometric Characteristics and Response Times for Content-Parallel Extended-Matching and One-Best-Answer Items in Relation to Number of Options

Swanson, David B.; Holtzman, Kathleen Z.; Allbee, Krista; Clauser, Brian E.

doi:10.1097/01.acm.0000236518.87708.9d

Cited by 31 publications

(22 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The IDn-IDc column therefore contains the difference between these two Item Difficulty values and the RTn -RTc column contains the difference in seconds between the total recorded response times for compliant and noncompliant items. The method chosen to calculate Item Difficulty (ID) for this experiment is the one used by (Swanson et al 2006). The psychometric characteristics used in (Swanson et al 2006) also include the Logit Transform of the ID.…”

Section: Resultsmentioning

confidence: 99%

“…The method chosen to calculate Item Difficulty (ID) for this experiment is the one used by (Swanson et al 2006). The psychometric characteristics used in (Swanson et al 2006) also include the Logit Transform of the ID. This is an attempt to compensate for the non-linear nature of the difficulty curve as ID values approach ID=1.0.…”

Section: Resultsmentioning

confidence: 99%

“…There is no need for this adjustment in the current study since the purpose is comparison of equivalent measures. The Item-Total bi-serial correlation coefficient for each of the altered MCQ test items is also referred to in (Swanson et al 2006). This measure is not usable because the calculation leads to 'divide by zero' errors when ID=1.0.…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Adapting Multiple-Choice Item-Writing Guidelines to an Industrial Context

Foster

2010

Proceedings of the 12th International Conference on Enterprise Information Systems

View full text Add to dashboard Cite

Abstract:This paper proposes a guideline for writing MCQ items for our domain which promotes the use of Multiple Alternative Choice (MAC) items. This guideline is derived from one of the guidelines from the Taxonomy of item-writing guidelines reviewed by Haladyna et al, 2002. The new guideline is tested by delivering two sets of MCQ test items to a representative sample of candidates from the domain. One set of items complies with the proposed guideline and the other set of items does not. Evaluation of the relative effectiveness of the items in the experiment is achieved using established methods of item response analysis. The experiment shows that the new guideline is more applicable to the featured domain than the original guideline.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Adapting Multiple-Choice Item-Writing Guidelines to an Industrial Context

Foster

2010

Proceedings of the 12th International Conference on Enterprise Information Systems

View full text Add to dashboard Cite

show abstract

“…The format allows for examination of a large number of cases in a short timeframe;Vignette-based multiple choice questions - short vignettes are presented and responses to judgements or decisions about any aspect of the case are called for from a list of options however these only assess the technical rational aspect of reasoning [36];Extended matching questions [37–39] - students are required to match a series of brief case scenarios based on a single complaint with the most appropriate diagnosis or action;Problem based scenarios that present aspects of a case in steps, each one requiring recording of student reasoning and investigation of knowledge sources for solutions to queries; andScript Concordance Tests (SCT) [40] - use case based scenarios with a series of questions prompting further diagnostic thought and analysis, conducted online or in person.…”

Section: Introductionmentioning

confidence: 99%

Developing a viva exam to assess clinical reasoning in pre-registration osteopathy students

et al. 2014

View full text Add to dashboard Cite

BackgroundClinical reasoning (CR) is a core capability for health practitioners. Assessing CR requires a suite of tools to encompass a wide scope of contexts and cognitive abilities. The aim of this project was to develop an oral examination and grading rubric for the assessment of CR in osteopathy, trial it with senior students in three accredited university programs in Australia and New Zealand, and to evaluate its content and face validity.MethodsExperienced osteopathic academics developed 20 cases and a grading rubric. Thirty senior students were recruited, 10 from each university. Twelve fourth year and 18 fifth year students participated. Three members of the research team were trained and examined students at an institution different from their own. Two cases were presented to each student participant in a series of vignettes. The rubric was constructed to follow a set of examiner questions that related to each attribute of CR. Data were analysed to explore differences in examiner marking, as well as relationships between cases, institutions, and different year levels. A non-examining member of the research team acted as an observer at each location.ResultsNo statistical difference was found between the total and single question scores, nor for the total scores between examiners. Significant differences were found between 4th and 5th students on total score and a number of single questions. The rubric was found to be internally consistent.ConclusionsA viva examination of clinical reasoning, trialled with senior osteopathy students, showed face and content validity. Results suggested that the viva exam may also differentiate between 4th and 5th year students’ capabilities in CR. Further work is required to establish the reliability of assessment, to further refine the rubric, and to train examiners before it is implemented as a high-stakes assessment in accredited osteopathy programs.Electronic supplementary materialThe online version of this article (doi:10.1186/1472-6920-14-193) contains supplementary material, which is available to authorized users.

show abstract

“…The conclusions listed in section 6, highlight the difficulties experienced by the researchers when comparing the efficacy [13], [14] of the CAREGen MCQ creation methodology with other systems that tackle the MCQ generation task [15], [16]. There does not appear to be an appropriate, quantitative measure of efficacy in the literature that might be used to evaluate the CAREGen methodology.…”

Section: Introductionmentioning

confidence: 99%

Using a new MCQ Generation methodology (CAREGen) in the UK Electricity Distribution Industry

Foster¹

2012

IJDS

View full text Add to dashboard Cite

show abstract

Psychometric Characteristics and Response Times for Content-Parallel Extended-Matching and One-Best-Answer Items in Relation to Number of Options

Abstract: Use of the extended-matching format and smaller numbers of options per item (and more items) should result in more efficient use of testing time and greater score precision per unit of testing time.

Cited by 31 publications

References 11 publications

Adapting Multiple-Choice Item-Writing Guidelines to an Industrial Context

Adapting Multiple-Choice Item-Writing Guidelines to an Industrial Context

Developing a viva exam to assess clinical reasoning in pre-registration osteopathy students

Using a new MCQ Generation methodology (CAREGen) in the UK Electricity Distribution Industry

Contact Info

Product

Resources

About