BackgroundA method for assessing the model validity of randomised controlled trials of homeopathy is needed. To date, only conventional standards for assessing intrinsic bias (internal validity) of trials have been invoked, with little recognition of the special characteristics of homeopathy. We aimed to identify relevant judgmental domains to use in assessing the model validity of homeopathic treatment (MVHT). We define MVHT as the extent to which a homeopathic intervention and the main measure of its outcome, as implemented in a randomised controlled trial (RCT), reflect 'state-of-the-art' homeopathic practice.MethodsUsing an iterative process, an international group of experts developed a set of six judgmental domains, with associated descriptive criteria. The domains address: (I) the rationale for the choice of the particular homeopathic intervention; (II) the homeopathic principles reflected in the intervention; (III) the extent of homeopathic practitioner input; (IV) the nature of the main outcome measure; (V) the capability of the main outcome measure to detect change; (VI) the length of follow-up to the endpoint of the study. Six papers reporting RCTs of homeopathy of varying design were randomly selected from the literature. A standard form was used to record each assessor's independent response per domain, using the optional verdicts 'Yes', 'Unclear', 'No'. Concordance among the eight verdicts per domain, across all six papers, was evaluated using the kappa (κ) statistic.ResultsThe six judgmental domains enabled MVHT to be assessed with 'fair' to 'almost perfect' concordance in each case. For the six RCTs examined, the method allowed MVHT to be classified overall as 'acceptable' in three, 'unclear' in two, and 'inadequate' in one.ConclusionFuture systematic reviews of RCTs in homeopathy should adopt the MVHT method as part of a complete appraisal of trial validity.

Thompson, E.A. et al. (2008) Toward standard setting for patientreported outcomes in the NHS homeopathic hospitals. Homeopathy, 97

These results do not support concern that deficient MVHT has frequently undermined the published findings of RCTs of individualised homeopathy. However, the 13 trials with 'uncertain' or 'inadequate' MVHT will be a focus of attention in supplementary meta-analysis. New RCTs of individualised homeopathy must aim to maximise MVHT and to enable its assessment through clear reporting.

Purpose : A new programme of systematic reviews of randomised controlled trials (RCTs) of homeopathy distinguishes several key attributes of study design and quality: placebo controlled cf. other-than-placebo controlled; individualised cf. non-individualised homeopathy; treatment cf. prophylaxis; internal validity cf. model validity. The present phase of the review programme focuses on assessing the model validity (MV) of peer-reviewed, placebo-controlled, RCTs of individualised homeopathic treatment. Methods : A systematic literature search and subsequent reappraisal of retrieved records identified 31 RCTs that satisfied the inclusion criteria for the present study. MV of the eligible RCTs was appraised using a novel criterion-based method. Assessment domains address: (i) the rationale for the choice of the particular homeopathic intervention; (ii) the homeopathic principles reflected in the intervention; (iii) the extent of homeopathic practitioner input; (iv) the relevance of the main outcome measure; (v) the capability of the main outcome measure to detect change; (vi) the length of follow-up to the endpoint of the study. These six MV domains per RCT were categorised by each of three independent assessors as ?acceptable?, ?unclear? or ?unacceptable?, disparities of opinion being resolved by consensus discussion. Results : Domain-specific and overall ratings of MV per RCT await the outcome of ongoing consensus discussions. A full set of findings will be presented at conference. Conclusions : MV data contribute importantly to the appraisal of RCT quality in systematic reviews of homeopathy.

a b s t r a c tBackground: To date, our programme of systematic reviews has assessed randomised controlled trials (RCTs) of individualised homeopathy separately for risk of bias (RoB) and for model validity of homeopathic treatment (MVHT). Objectives: The purpose of the present paper was to bring together our published RoB and MVHT findings and, using an approach based on GRADE methods, to merge the quality appraisals of these same RCTs, examining the impact on meta-analysis results. Design: Systematic review with meta-analysis. Methods: As previously, 31 papers (reporting a total of 32 RCTs) were eligible for systematic review and were the subject of study. Main outcome measures: For each trial, the separate ratings for RoB and MVHT were merged to obtain a single overall quality designation ('high', 'moderate, "low", 'very low'), based on the GRADE principle of 'downgrading'. Results: Merging the assessment of MVHT and RoB identified three trials of 'high quality', eight of 'moderate quality', 18 of 'low quality' and three of 'very low quality'. There was no association between a trial's MVHT and its RoB or its direction of treatment effect (P > 0.05). The three 'high quality' trials were those already labelled 'reliable evidence' based on RoB, and so no change was found in meta-analysis based on best-quality evidence: a small, statistically significant, effect favouring homeopathy. Conclusion: Accommodating MVHT in overall quality designation of RCTs has not modified our preexisting conclusion that the medicines prescribed in individualised homeopathy may have small, specific, treatment effects.