Development of a scale to measure practitioner adherence to a brief intervention in the emergency department

Abstract

Brief intervention (BI) can reduce harmful and hazardous drinking among emergency department patients. However, no psychometrically-validated instrument for evaluating the extent to which practitioners correctly implement BIs in clinical practice (e.g., adherence) exists. We developed and subsequently examined the psychometric properties of a scale that measures practitioner adherence to a BI, namely the Brief Negotiation Interview (BNI). Ratings of 342 audio-taped BIs in the emergency department demonstrated that the BNI Adherence Scale (BAS) has: 1) excellent internal consistency and discriminant validity; 2) good to excellent inter-rater reliability, and 3) good construct validity, with an 8-item, 2-factor structure accounting for 62% of the variance, but 4) no predictive validity in this study. The BAS provides practitioners with a brief, objective method to evaluate their BNI skills and give feedback to them about their performance.

Development of the SBIRT checklist for observation in real‐time (SCORe)

“…First, because the procedure was designed for ease of use, it assesses adherence, and does not necessarily measure competence or how skillfully particular elements are delivered. Although competence assumes adherence, it may affect outcomes separately [11]. Secondly, the brief treatment component of SAMHSA-funded SBIRT programs was not evaluated.…”

“…Three of the original 16 screening items [10,12,16] and four of the original 26 BI items (exercise 1: 1, 7, 26; exercise 2: 4, 7, 26) were excluded. Total agreement scores were based on the percentage of components identified correctly as present or absent across the two certification exercises, the percentage agreement to the BI content components [1][2][3][4][5][6][7][8][9][10][11][12][13][14]25] and percentage agreement to the MI style/spirit elements [15][16][17][18][19][20][21][22][23][24]. At least two provider occurrences of the behavior were required to meet the adherence threshold for MI spirit/style elements.…”

“…Until recently, there have been no empirically validated instruments for assessing SBIRT adherence [11], i.e. the extent to which specific, pre-defined activities are utilized as specified by evidence-based protocols.…”

“…Finally, a methodology to assess adherence is needed to facilitate SBIRT implementation and dissemination research [11]. Adherence has been identified as an important implementation outcome for SBIRT program evaluation [4].…”

Readiness to Change and to Accept Help and Drinking Outcomes in Young Adults of Mexican Origin

“…To assure fidelity, interventions were initially observed with patient consent, later taped for feedback from experts. Last, adherence was assessed using a tested and validated scoring system (Pantalon et al, 2012).…”