Reliability in multi‐site psychiatric studies

Leese, Morven; White, Ian R.; Schene, Aart H.; Koeter, Maarten W. J.; Ruggeri, Mirella; Gaite, L.

doi:10.1002/mpr.98

Cited by 6 publications

(6 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Intra-class correlations (ICCs) were calculated, using a two-way mixed model defi ning agreement in terms of consistency, to assess both inter-rater and test-retest reliability for continuous scores (Bland and Altman, 1986) (need versus no need and unmet need versus met or no need). The ICCs represent the ratio of the variance of the true score between subjects and the total variance (Leese et al, 2001). Analyses were carried out in the Statistical Package for Social Sciences (SPSS version 15.0, 2006).…”

Section: Discussionmentioning

confidence: 99%

The reliability and validity of the forensic Camberwell Assessment of Need (CANFOR): a needs assessment for forensic mental health service users

Thomas

Slade

McCrone

et al. 2008

Int J Methods Psych Res

110

View full text Add to dashboard Cite

No instrument exists that measures the individual needs of forensic mental health service users (FMHSUs). The aim of this study was therefore to develop a valid and reliable individual needs assessment instrument for FMHSUs that incorporated staff and service user views and measured met and unmet needs. The Camberwell Assessment of Need was used as a template to develop CANFOR. Consensual and content validity were investigated with 50 forensic mental health professionals and 60 FMHSUs. Both were found to be satisfactory. Concurrent validity was tested using the Global Assessment of Functioning and a five-point needs scale, and again was found to be satisfactory. Reliability studies were carried out with 77 service users and 65 staff in high and medium security psychiatric services in the UK. Inter-rater reliability, rating whether a need was present or not, was high for service users (0.991) and staff (0.998). Similarly high reliability was found for unmet needs (0.985 and 0.972, respectively). Test-retest reliability was found to be moderately high for service users (0.795) and staff (0.852) when ratings were made two weeks apart. Similar levels were found for ratings of unmet needs (0.813 and 0.699, respectively). The average interview time was 23 minutes. CANFOR has good validity and reliability, and is suitable for further testing with other service user groups.

show abstract

Section: Discussionmentioning

confidence: 99%

The reliability and validity of the forensic Camberwell Assessment of Need (CANFOR): a needs assessment for forensic mental health service users

Thomas

Slade

McCrone

et al. 2008

Int J Methods Psych Res

110

View full text Add to dashboard Cite

show abstract

“…Intra-class correlation coefficients (ICC) were calculated, using a two-way mixed model defining, to assess inter-rater reliability for the CANFOR total needs, total met needs and total unmet needs, as recommended by Leese (2001) (60). Cohen's Kappa coefficients were calculated to assess the level of agreement on each need domain between patients and clinicians.…”

Section: Analysesmentioning

confidence: 99%

Exploring Needs and Quality of Life of Forensic Psychiatric Inpatients in the Reformed Italian System, Implications for Care and Safety

Vorstenbosch

Castelletti

2020

Front. Psychiatry

View full text Add to dashboard Cite

The Italian forensic psychiatric system underwent drastic reforms. The newly developed facilities are inspired by psychiatric community services, embracing a recovery-oriented approach. Needs and quality of life are broader concepts that consider the more rehabilitative and humanitarian aspects of treatment. In one of the new Italian forensic psychiatric services, this cross-sectional study aimed to investigate the needs and quality of life of forensic psychiatric patients. A second aim was to validate the Italian version of the Forensic inpatient Quality of Life questionnaire Short Version (FQL-SV). Overall, 42 forensic psychiatric patients were assessed using the Forensic version of the Camberwell Assessment of Need (CANFOR), the Historical-Clinical-Risk-Management-20 (HCR-20), the FQL-SV, and the World Health Organization Quality of Life (WHOQoL-Bref). Patients reported significantly fewer needs, whether met or unmet, than their treating clinicians. The general level of agreement between patients and clinicians on specific needs was low Kappa values were < .40 for 64% of the total needs and 46% of the unmet needs. Risk factors according to the HCR-20 mean scores were 13.1, 4.6, and 6.4 for the historical, clinical and risk management subscale. Quality of life was moderate to high for 74% of the patients. Our results showed that lower numbers of needs, whether reported by patients or clinicians, were associated with a better quality of life. The Italian FQL-SV had a Cronbach's alpha of 0.86 and correlated as expected with the WHOQoL-Bref. The FQL-SV is a valid and reliable tool, justifying its use for routinely assessing QoL in Italian forensic psychiatric services. This study enhances our understanding of needs and quality of life of forensic psychiatric patients and how their assessment could have an additional value for recoveryoriented treatment in forensic psychiatry. Although the detained status of forensic patients imposes real limits on the capacity for autonomy and choice, incorporating the patient's perspective on decision-making processes, in relation to aspects of treatment, care, and daily life, may have benefits such as a better treatment adherence or therapeutic alliance. Future research should clarify how routinely assessing needs and quality of life can contribute to the recovery of these forensic psychiatric patients.

show abstract

“…According to the definition used by Roncone et al (1999), inter-rater reliabilities ≥ +0.75 were considered to be 'good,' inter-rater reliabilities < +0.50 were considered to be 'poor,' and intermediate values were regarded as 'acceptable.' Thus, these guidelines are more stringent than those previously proposed by Cicchetti et al (1992) or Leese et al (2001).…”

Section: Statistical Analysesmentioning

confidence: 88%

“…Therefore, the need to establish and maintain high cross-national inter-rater reliability on specific instruments will continue (cf. Shrout, 1998;Leese et al, 2001). With few resources from research grants to allow proper cross-national adaptation processes as conducted in the EPSILON study (Knudsen et al, 2000), more economical processes for successful implementation are of particular interest.…”

Section: Introductionmentioning

confidence: 97%

“…Currently, very few measures are available with standardized translations to ensure their cross-national applicability (Knudsen et al, 2000). In response to this, the EPSILON study (Becker et al, 2000;Leese et al, 2001) recently adapted five key instruments for the assessment of needs, service utilization and costs, informal carer involvement, quality of life, and service satisfaction, for use in five languages (Danish, Dutch, English, Italian and Spanish). Further dimensions should also be assessed in psychiatric service evaluation, however (Hansson, 2001), and future multi-site studies will increasingly involve other European languages (for example, Becker et al, 2004).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Inter‐rater reliability of the Brief Psychiatric Rating Scale and the Groningen Social Disabilities Schedule in a European multi‐site randomized controlled trial on the effectiveness of acute psychiatric day hospitals

Schützwohl

Jarosz-Nowak

Briscoe

et al. 2003

Int J Methods Psych Res

View full text Add to dashboard Cite

The objectives of this study were to report the inter-rater reliability of the Brief Psychiatric Rating Scale (BPRS 4.0) and the Groningen Social Disabilities Schedule (GSDS-II) as assessed in a randomized controlled trial on the effectiveness of psychiatric day hospitals spanning five sites in countries of Central and Western Europe. Following brief training sessions, videotaped BPRS-interviews and written GSDS-vignettes were rated by clinically experienced researchers from all participating sites. Inter-rater reliability often proved to be poor for items assessing the severity of both psychopathology and social dysfunction, but findings suggest that both instruments allow for the assessment of the presence or absence of specific psychopathological symptoms or social disabilities. Inter-rater reliability at subscale level proved to be good for both instruments. Results indicate that, with a brief training session and proper use of the instruments, psychopathology and social disabilities can be reliably assessed within cross-national research studies. The results are of particular interest given that the need to conduct cross-national multi-site studies including countries with different cultural backgrounds increases.

show abstract

Reliability in multi‐site psychiatric studies

Cited by 6 publications

References 34 publications

The reliability and validity of the forensic Camberwell Assessment of Need (CANFOR): a needs assessment for forensic mental health service users

The reliability and validity of the forensic Camberwell Assessment of Need (CANFOR): a needs assessment for forensic mental health service users

Exploring Needs and Quality of Life of Forensic Psychiatric Inpatients in the Reformed Italian System, Implications for Care and Safety

Inter‐rater reliability of the Brief Psychiatric Rating Scale and the Groningen Social Disabilities Schedule in a European multi‐site randomized controlled trial on the effectiveness of acute psychiatric day hospitals

Contact Info

Product

Resources

About