Measuring individual significant change on the Beck Depression Inventory-II through IRT-based statistics

Abstract

Several researchers have emphasized that item response theory (IRT)-based methods should be preferred over classical approaches in measuring change for individual patients. In the present study we discuss and evaluate the use of IRT-based statistics to measure statistical significant individual change on the Beck Depression Inventory-II (BDI-II, Beck, Steer, & Brown, 1996). We compare results obtained with a simple IRT-based statistical test (Z-test) to those obtained with the Reliable Change Index (RCI) in a sample of clinical outpatients. Mean group differences between the Z-test and the RCI were similar, but for some individuals change classifications differed. Differences were most evident for change scores within the lower range of depression scores. We show that this may have consequences for the measurement of individual change in psychotherapy outcome research and clinical practice.

Diagnostic Utility and Psychometric Properties of the Beck Depression Inventory-II Among Korean Adults

“…IRT analysis suggested that BDI-II could offer equivalent information value from an average depressed population (where θ is 0) to a severely depressed population (where θ is 3), which is excellent for a severity rating tool, as mentioned earlier. This result is in line with a previous study (Brouwer et al, 2013) that conducted an IRT analysis of the BDI-II. A study by Brouwer et al (2013) reported flat-looking TIC graphs for the range θ = 0-2, and argued that this may be more advantageous for detecting changes in depression in the clinical field.…”

“…This result is in line with a previous study (Brouwer et al, 2013) that conducted an IRT analysis of the BDI-II. A study by Brouwer et al (2013) reported flat-looking TIC graphs for the range θ = 0-2, and argued that this may be more advantageous for detecting changes in depression in the clinical field. These results of the IRT analysis of BDI-II are consistent with the original intent of developing the BDI-II scale, which was to measure the depth of depression rather than simply presenting a single cut-off point (Beck et al, 1996).…”

Unidimensional Item Response Theory

“…The models are less suited to the construction of computer adaptive tests or when using change scores. Several authors have discussed that change scores are more difficult to interpret using total scores than when using parametric IRT scoring (Brouwer, Meijer, & Zevalkink, 2013;Embretson & Reise, 2000;Reise & Haviland, 2005). A general guide in deciding which model to apply is that nonparametric IRT is an interesting tool to explore data quality, however when trait estimates are needed parametric models must be used.…”

Time-Efficient Adaptive Measurement of Change

“…Indeed, the measurement of individual change has been studied among chiropractic patients with chronic neck or low back pain (Hays, Spritzer, Sherbourne, Ryan, & Coulter, 2019), patients undergoing foot and ankle treatment (Hung et al, 2019), and children with chronic pain (Kashikar-Zuck et al, 2016). Psychological assessment may also be conducted on more than one occasion for a given respondent to determine whether a significant change in their beliefs, mental health, or behavior has occurred (Brouwer, Meijer, & Zevalkink, 2013;Kruyen, Emons, & Sijtsma, 2014;Wang & Weiss, 2018). In the field of education, testing might be repeated to gauge a student's degree of improvement (or decline) in a given domain over time (Wang & Weiss, 2018;Weiss & Von Minden, 2011).…”

“…In medical contexts, Hays et al (2019) used a three-month follow-up, whereas Hung et al (2019) stratified their results by followup time, with three months being the shortest follow-up and a period of over six months being the longest. In the contexts of psychological and educational assessment, respectively, Brouwer et al (2013) and Wang and Weiss (2018) both examined results after a follow-up time of approximately one year.…”