Assessing Family Functioning: Psychometric Evaluation of the Family Star Plus

Abstract

Objective To examine the psychometric properties of the Family Star Plus, an assessment of family functioning using a collaborative mode of completion between the caseworker and service user. Background The Family Star Plus is widely used to assess strengths and needs within services supporting families. Method In Phase 1, interrater reliability was assessed. Phase 2 established factor structure, internal consistency, item redundancy, and responsiveness. In Phase 3, the Family Star Plus was used to predict children's unauthorized school absence. Results Workers reliably applied Family Star Plus scales. A unidimensional factor structure was found with good internal consistency and responsiveness. School absence was predicted by the Family Star Plus. Conclusions These findings demonstrate the validity of the Family Star Plus as a tool for assessing needs and measuring progress of family support interventions. Implications The Family Star Plus can be confidently used and recommended as an outcomes measurement tool.

Unsubstantiated conclusions about the Family Star Plus as an outcome measure: a rebuttal to Sweet, Winter, Neeson and Connolly (2020)

“…They also focus on pilot findings reported in the study of MacKeith (2014) from a much earlier precursor of the Family Star Plus (the first edition of the Family Star), describing agreement on the journey of change in negative terms, when it fact it was substantially higher than frequently cited thresholds (Chaturvedi and Shweta, 2015). Rather surprisingly, they did not mention the inter-rater reliability findings shared with them in manuscript form prior to the publication of Good and MacKeith (2020). Good and MacKeith (2020) showed very good inter-rater reliability for the Family Star Plus (Krippendorff’s α = 0.83), using a much larger sample than MacKeith (2014) and with a chance-corrected inter-rater reliability coefficient.…”

“…Rather surprisingly, they did not mention the inter-rater reliability findings shared with them in manuscript form prior to the publication of Good and MacKeith (2020). Good and MacKeith (2020) showed very good inter-rater reliability for the Family Star Plus (Krippendorff’s α = 0.83), using a much larger sample than MacKeith (2014) and with a chance-corrected inter-rater reliability coefficient. This article also demonstrated that Star readings predicted hard outcomes, offering further evidence that they can be meaningful and accurately completed.…”

“…Secondly, principal components analysis was not designed for ordinal data (Overall, 1964) and has been shown to overestimate the number of dimensions (Hubbard and Allen, 1987; Ruscio and Roche, 2012; Zwick and Velicer, 1986). Good and MacKeith’s (2020) analyses (parallel analysis followed by confirmatory factor analysis) used a much larger sample and found the Family Star Plus to be unidimensional (Good and MacKeith, 2020). Parallel analysis (Horn, 1965) is appropriate for ordinal data and has been found to be superior to conventional methods for correctly identifying the number of dimensions (Hubbard and Allen, 1987; Ruscio and Roche, 2012; Zwick and Velicer, 1986).…”

“…Sweet and colleagues state that “further work is required before the Family Star Plus can be considered for use as an outcomes measure”. Leaving aside the issues identified with their analyses and conclusions, further work using a much larger sample and more comprehensive and appropriate testing has now been published in a peer-reviewed article (Good and MacKeith, 2020). This paper presents evidence that the Family Star Plus has good inter-rater reliability, internal consistency, a unidimensional factor structure, no item redundancy, is responsiveness and has predictive validity.…”

“…Means are only used when continuous data is analysed (e.g. averaging the inter-rater reliability statistic), and very similar methods are used, those reported in a recent peer-reviewed validation article for the Family Star Plus (Good and MacKeith, 2020). In addition, there is a growing body of externally conducted evidence for the validity of the Stars, which is available through the usual search methods as well as being collated in Triangle’s Research Library (www.outcomesstar.org.uk/about-the-star/evidence-and-research/research-library/#all).…”