Learning to mark: a qualitative study of the experiences and concerns of medical markers

Hawthorne, Kamila; Wood, Fiona; Hood, Kerenza; Cannings‐John, Rebecca; Houston, Helen

doi:10.1186/1472-6920-6-25

Cited by 4 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Figs. 2 and 3 (and in Tables 4 and 5 ) above, the use of a ‘single screening’ approach would have resulted in the exclusion of one eligible study [ 22 ] from the ‘case study’ systematic review, while use of the ‘single screening with text mining’ approach would have resulted in the exclusion of eight other eligible studies [ 23 – 30 ]. Analysis of the contributions made by these nine ‘false negative’ studies to the ‘case study’ review found that all nine contributed only to the descriptive component of the review (i.e.…”

Section: Resultsmentioning

confidence: 99%

Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews

Shemilt¹,

et al. 2016

View full text Add to dashboard Cite

BackgroundMeta-research studies investigating methods, systems, and processes designed to improve the efficiency of systematic review workflows can contribute to building an evidence base that can help to increase value and reduce waste in research. This study demonstrates the use of an economic evaluation framework to compare the costs and effects of four variant approaches to identifying eligible studies for consideration in systematic reviews.MethodsA cost-effectiveness analysis was conducted using a basic decision-analytic model, to compare the relative efficiency of ‘safety first’, ‘double screening’, ‘single screening’ and ‘single screening with text mining’ approaches in the title-abstract screening stage of a ‘case study’ systematic review about undergraduate medical education in UK general practice settings. Incremental cost-effectiveness ratios (ICERs) were calculated as the ‘incremental cost per citation ‘saved’ from inappropriate exclusion’ from the review. Resource use and effect parameters were estimated based on retrospective analysis of ‘review process’ meta-data curated alongside the ‘case study’ review, in conjunction with retrospective simulation studies to model the integrated use of text mining. Unit cost parameters were estimated based on the ‘case study’ review’s project budget. A base case analysis was conducted, with deterministic sensitivity analyses to investigate the impact of variations in values of key parameters.ResultsUse of ‘single screening with text mining’ would have resulted in title-abstract screening workload reductions (base case analysis) of >60 % compared with other approaches. Across modelled scenarios, the ‘safety first’ approach was, consistently, equally effective and less costly than conventional ‘double screening’. Compared with ‘single screening with text mining’, estimated ICERs for the two non-dominated approaches (base case analyses) ranged from £1975 (‘single screening’ without a ‘provisionally included’ code) to £4427 (‘safety first’ with a ‘provisionally included’ code) per citation ‘saved’. Patterns of results were consistent between base case and sensitivity analyses.ConclusionsAlternatives to the conventional ‘double screening’ approach, integrating text mining, warrant further consideration as potentially more efficient approaches to identifying eligible studies for systematic reviews. Comparable economic evaluations conducted using other systematic review datasets are needed to determine the generalisability of these findings and to build an evidence base to inform guidance for review authors.

show abstract

Section: Resultsmentioning

confidence: 99%

Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews

Shemilt¹,

et al. 2016

View full text Add to dashboard Cite

show abstract

“…It is possible that the new task of formulating feedback in writing made the teaching more challenging. It has not only been found that teachers lack self-confidence in making assessments and giving feedback, but also that the rapid pace of ambulatory clinics reduces the opportunities for teaching and learning (Bowen & Irby 2002;Hawthorne et al 2006). Possibly, the StudentPEP project did not take the teachers' busy work in general practice seriously enough.…”

Section: Teachers' Feedbackmentioning

confidence: 99%

Triangulation of written assessments from patients, teachers and students: Useful for students and teachers?

2010

View full text Add to dashboard Cite

Background: Many medical students in general practice clerkships experience lack of observation-based feedback. The StudentPEP project combined written feedback from patients, observing teachers and students. Aim: This study analyzes the perceived usefulness of triangulated written feedback. Methods: A total of 71 general practitioners and 79 medical students at the University of Oslo completed project evaluation forms after a 6-week clerkship. A principal component analysis was performed to find structures within the questionnaire. Regression analysis was performed regarding students' answers to whether StudentPEP was worthwhile. Free-text answers were analyzed qualitatively.Results: Student and teacher responses were mixed within six subscales, with highest agreement on 'Teachers oral and written feedback' and 'Attitude to patient evaluation'. Fifty-four per cent of the students agreed that the triangulation gave concrete feedback on their weaknesses, and 59% valued the teachers' feedback provided. Two statements regarding the teacher's attitudes towards StudentPEP were significantly associated with the student's perception of worthwhileness. Qualitative analysis showed that patient evaluations were encouraging or distrusted. Some students thought that StudentPEP ensured observation and feedback. Conclusion: The patient evaluations increased the students' awareness of the patient perspective. A majority of the students considered the triangulated written feedback beneficial, although time-consuming. The teacher's attitudes strongly influenced how the students perceived the usefulness of StudentPEP.

show abstract

Research project assessments and supervisor marking: maintaining academic rigour through robust reconciliation processes

McQuade

Kometa

Brown

et al. 2020

Assessment & Evaluation in Higher Education

View full text Add to dashboard Cite

Learning to mark: a qualitative study of the experiences and concerns of medical markers

Cited by 4 publications

References 15 publications

Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews

Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews

Triangulation of written assessments from patients, teachers and students: Useful for students and teachers?

Research project assessments and supervisor marking: maintaining academic rigour through robust reconciliation processes

Contact Info

Product

Resources

About