Some chondrocytes are much more vulnerable to repetitive mechanical loading than others, suggesting that vigorous activity may lead to cell death in articular cartilage.

SummaryNon-technical skills are recognised as crucial to good anaesthetic practice. We designed and evaluated a specialty-specific tool to assess non-technical aspects of trainee performance in theatre, based on a system previously found reliable in a recruitment setting. We compared inter-rater agreement (multir-ater kappa) for live assessments in theatre with that in a selection centre and a video-based rater training exercise. Twenty-seven trainees participated in the first in-theatre assessment round and 40 in the second. Round-1 scores had poor inter-rater agreement (mean kappa = 0.20) and low reliability (generalisability coefficient G = 0.50). A subsequent assessor training exercise showed good inter-rater agreement, (mean kappa = 0.79) but did not improve performance of the assessment tool when used in round 2 (mean kappa = 0.14, G = 0.42). Inter-rater agreement in two selection centres (mean kappa = 0.61 and 0.69) exceeded that found in theatre. Assessment tools that perform reliably in controlled settings may not do so in the workplace. The Tooke report on Modernising Medical Careers [1] highlighted the need for specialty training to focus on the acquisition of excellence, rather than competence alone. Recent editorials have attempted to define how excellence in professionalism and other domains manifest in the workplace and highlight the importance of non-technical skills [2,3]. Workplace-based assessments are an invaluable tool for assessing professional practice in a comprehensive and valid way; however, only the mini Clinical Evaluation Exercise (mini-CEX) and multi-source feedback amongst the assessment tools currently used in the UK attempt to assess nontechnical skills. In addition, these tools focus on the achievement of basic clinical competence and employ methods with questionable accuracy, reliability and validity [4]. The mini-CEX has been shown to have wide inter-rater variability that results in poor discrimination between anaesthetic trainees [5], that is exacerbated by the lack of performance benchmarking and behavioural descriptors on the marking sheet. Variable scoring leniency and the face-toface nature of the assessment may also contribute to inaccurate scores [5,6]. Studies have established the value of multi-source feedback in certain settings [7], although concerns have been raised about victimisation by multisource feedback raters [8]. These current workplace-based assessment tools have been described as stressful, timeconsuming, artificial and difficult to organise [8], and rely on immediate access to an electronic portfolio in many specialties. Large numbers of assessors are required for each trainee to achieve a reliable assessment, suggesting that the feasibility of their use in high-stakes assessment is low [9]. In addition, students are able to select individual cases, case difficulty and specific assessors, despite evidence that the relationship between the observer and student may adversely influence the validity of the assessment [6,10,11]. None of the tools described a...

-Clinical problem solving tests (CPSTs) have been shown to be reliable and valid for recruitment to general practice (GP) training programmes. This article presents the results from a Department of Health-funded pilot into the use of a CPST designed for recruitment to the acute specialties (AS). The pilot paper consisted of 99 items from the validated GP question bank and 40 new items aimed specifically at topics of relevance to AS training. The CPST successfully differentiated between applicants. The overall test and the GP section showed high internal reliability, whereas the AS pilot section performed less well. A detailed item analysis revealed that the AS pilot items were, on average, more difficult and of poorer quality than the GP items. Important issues that need to be addressed in the early development phase of a test used for high stakes selection to specialty training programmes are discussed. Subsequent research has shown the machine marked tests (MMTs) used in GP selection are as valid and reliable as the current standard shortlisting method of scoring application form questions. 2 In light of this, this article presents the preliminary results from an ongoing Department of Health-funded pilot study of the development of a clinical problem solving test (CPST) tailored to the acute medical specialties, looking specifically at its reliability, content validity and face validity. The initial pilot testThe CPST paper consisted of 139 items mapped onto the foundation programme curriculum: 99 taken from the GP bank of validated items and 40 newly written items aimed specifically at topics of relevance to acute specialty (AS) training. The tried and tested GP items provided a useful benchmark against which to assess the performance of the newly developed AS items. The new items were written by a group of subject matter experts, including consultants in acute medicine, anaesthesia, emergency medicine and intensive care medicine.Applicants attending selection centres in 2008 at the South West Peninsula Deanery for CT1 training posts in anaesthesia, core medical training and the acute care common stem (ACCS) were invited to sit the first pilot CPST. Participation (or not) did not influence selection outcome in any way and participants consented to the linking of test results with other personal data. The paper was administered on the day of interview and an overall response rate of 74% (125 of 169 applicants) was achieved. Reliability of the testThe distribution of test scores was approximately normal in each section (GP and AS), indicating an absence of ceiling or floor effects and showing that the test has the potential to differentiate between candidates. Both the overall test and the GP item section showed very high internal reliability (Cronbach's ␣ϭ0.90 and 0.92 respectively). The AS pilot section performed less well (␣ϭ0.43), though this is partly attributable to the smaller number of items used. When corrected for test length (Spearman-Brown), the expected reliability of a 99-item test of equ...

Text datasets come in an abundance of shapes, sizes and styles. However, determining what factors limit classification accuracy remains a difficult task which is still the subject of intensive research. Using a challenging UK National Health Service (NHS) dataset, which contains many characteristics known to increase the complexity of classification, we propose an innovative classification pipeline. This pipeline switches between different text pre-processing, scoring and classification techniques during execution. Using this flexible pipeline, a high level of accuracy has been achieved in the classification of a range of datasets, attaining a micro-averaged F1 score of 93.30% on the Reuters-21578 “ApteMod” corpus. An evaluation of this flexible pipeline was carried out using a variety of complex datasets compared against an unsupervised clustering approach. The paper describes how classification accuracy is impacted by an unbalanced category distribution, the rare use of generic terms and the subjective nature of manual human classification.