The impact of item wording and behavioral specificity on the accuracy of direct behavior ratings (DBRs).

Riley‐Tillman, T. Chris; Chafouleas, Sandra M.; Christ, Theodore J.; Briesch, Amy M.; LeBel, Teresa J.

doi:10.1037/a0015248

Cited by 36 publications

(36 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, the wording of items on the DBR may have resulted in greater error. The behavior rated in the current study was specific (i.e., out‐of‐seat behavior), whereas previous research has found ratings of disruptive behavior to be more accurate when worded globally (e.g., disruptive behavior; Riley‐Tillman, Chafouleas, Christ, Briesch, & LeBel, ). Future researchers should consider evaluating the effects of completion latency when items are worded globally, as this may result in increased accuracy in ratings.…”

Section: Discussioncontrasting

confidence: 76%

“…Given that the purpose of the study was to examine the relative accuracy of different DBR completion latencies, the primary dependent variable used in the analyses was a measure of rating accuracy. Difference scores have been used in psychological research to demonstrate discrepancies between observed scores and a criterion (Mullins & Force, ), and have recently been used within the context of school‐based behavior assessment to examine the accuracy of teacher‐completed DBRs (Riley‐Tillman, Chafouleas, Christ, Briesch, & LeBel, ; Schlientz, Riley‐Tillman, Briesch, Walcott, & Chafouleas, ). In this study, difference scores were calculated by subtracting each participants’ DBR score from 40% (i.e., the criterion established through SDO) and using the absolute value of the difference.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Accuracy in direct behavior rating is minimally impacted by completion latency

Dart

Radley

Fischer

et al. 2017

Psychology in the Schools

View full text Add to dashboard Cite

Direct behavior ratings (DBRs) have been proposed as an efficient method to assess student behavior in the classroom due to their relative ease of administration compared to alternative methods like systematic direct observation. DBRs are considered low-inference assessments of behavior because they are designed to be completed immediately following a specified observation period of student behavior; however, in practice it is common for teachers and other respondents to delay completion of a DBR until they are reminded to do so. It is unclear what effect, if any, this latency between observation and DBR completion has on rater accuracy. Thus, the purpose of this study was to examine the effect of completion latency on accuracy in an analogue setting. Two-hundred forty-one undergraduate students (83.8% female) with a mean age of 21 participated across eight groups and were asked to complete an electronic DBR immediately after watching a video of student behavior or after a predetermined delay of 5 minutes, 15 minutes, 30 minutes, 1 hour, 2 hours, 4 hours, or 6 hours. A one-way analysis of variance revealed that there was no statistically significant relationship between completion latency and DBR accuracy, F(7, 233) = .959, p = .46, 2 = .028. K E Y W O R D Sbehavior assessment, completion latency, direct behavior rating Multitiered systems of support (MTSS) refers to a service delivery model in which academic, behavior, or mental health intervention services are provided to students based on their individual needs (e.g., Sugai & Horner, 2009). One tenet of MTSS involves frequent formative assessments of students' functioning to identify students that may require more intense intervention services than what is currently being provided. Additionally, continuous progress monitoring of those intervention services is necessary to make decisions about their effectiveness. With the rise of MTSS in schools, there is an increased need for universal screening and progress monitoring tools for the purposes of prevention and early identification of academic and behavior problems. In particular, these tools must be technically adequate,

show abstract

Section: Discussioncontrasting

confidence: 76%

Section: Methodsmentioning

confidence: 99%

Accuracy in direct behavior rating is minimally impacted by completion latency

Dart

Radley

Fischer

et al. 2017

Psychology in the Schools

View full text Add to dashboard Cite

show abstract

“…A measure of perceptions of their classmates' disruptive classroom behaviors was developed from the DBR-SIS (see above). Specific behaviors that comprised the disruptive behavior category were generated from previous DBR-SIS research (Riley-Tillman et al, 2009 ; Christ et al, 2011 ), and wording of the items was simplified to be appropriate for the age of the children. The eight disruptive acts presented to children were: “gets out of his/her seat without permission,” “talks or yells about things we're not working on,” “makes sounds (like humming, laughing, whistling) that aren't allowed during class time,” “talks to other kids when we're not allowed to,” “calls out things to the teacher without permission to talk,” “does or says things that interrupt what we're doing,” “is rude or mean to the teacher,” and “plays with things at his or her desk that don't have anything to do with our work.” Children were asked to rate each of their peers on each behavior using a 3-point response scale of “never,” “sometimes,” and “a lot/always.” The scale was administered to each child individually with the help of a graduate assistant and was completed over contiguous 3-day sessions.…”

Section: Methodsmentioning

confidence: 99%

“…In each grade, teachers were asked to rate students' classroom behaviors using the DBR-SIS (Direct Behavior Rating—Single Item Scales; Riley-Tillman et al, 2009 ; Chafouleas, 2011 ) following four 2-h instructional sessions toward the end of the school year. At the end of each class period teachers were asked to estimate how often each student showed disruptive behavior on a 5-point scale (1 = “never/almost never,” 3 = “sometimes,” 5 = “always/almost always”).…”

Section: Methodsmentioning

confidence: 99%

The Education of Playful Boys: Class Clowns in the Classroom

Barnett

2018

Front. Psychol.

View full text Add to dashboard Cite

This longitudinal study identified degrees of playfulness in 278 kindergarten-aged children, and followed them through their next three school years to determine how playfulness was viewed by the children themselves, their classmates, and teachers. Perceptions of the social competence, disruptiveness, and labeling as the class clown, were assessed from all perspectives in each of first through third grades. Hierarchical linear modeling was conducted to account for the nesting of the data (children within classrooms within schools) and for the lack of independence between the measures. A central finding confirmed extant literature in that gender differences were dominant, with playful boys regarded as distinct from their less playful counterparts, while no such discrepancies appeared for girls. Playful boys were increasingly negatively regarded as rebellious and intrusive and were labeled as the “class clown” by their teachers. These findings were in direct contrast with children's self-perceptions and those of their peers, who initially regarded more playful boys as appealing and engaging playmates. The data further revealed that the playful boys were stigmatized by their teachers, and this was communicated through verbal and non-verbal reprimands, and classmates assimilated this message and became increasingly denigrating of the playful quality in the boys. In stark contrast, girls' playfulness levels were not a consideration in ratings by teachers or peers at any grade, nor did their classroom behaviors show significant variation. These negative perceptions were likely transferred by teachers to peers and to the children themselves, whereupon they changed their positive perceptions to be increasingly negative by third grade. The results contribute to the literature by demonstrating that playfulness in boys (but not girls) is often associated with the “class clown” designation, and is viewed as an increasingly lethal characteristic in school classrooms, where compelling efforts are undertaken to discourage its expression and persistence.

show abstract

“…Over the past few years, Chafouleas, Riley-Tillman, and colleagues have initiated a systematic evaluation of DBR-SIS that includes considerations such as behavior targets, rating procedures, scale design, and comparisons with other methods. For example, one line of investigations involved the influence of specificity (being molar versus molecular) and phrasing (being positive versus negative) on the accuracy of target behavior rating (Chafouleas et al, Submitted for publication;Christ, Riley-Tillman, Chafouleas, & Jaffery, 2011;Riley-Tillman, Chafouleas, Christ, Briesch, & LeBel, 2009). Generally, results of this research suggested that wording of target behaviors used in DBR-SIS should be described globally rather than as specific indicators to enhance rating outcomes associated with reliability and accuracy.…”

Section: Assessment Of Behavior Using Direct Behavior Ratingmentioning

confidence: 99%