Interrater Reliability of Therapists' Judgments of Graphed Data

Harbst, Kimberly; Ottenbacher, Kenneth J.; Harris, Susan R.

doi:10.1093/ptj/71.2.107

Cited by 50 publications

(37 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers have recommended that in single-subject designs the use of both visual inspection and statistical analysis methods provides complementary information related to treatment impact (Kazdin, 1982;Harbst et al, 1991;Kratochwill and Levine, 1992;Nourbakhsh and Ottenbacher, 1994). Statisti-cal analysis is particularly useful when changes appear to be small on visual inspection (Kazdin, 1982;Wolery and Harris, 1982;Ottenbacher, 1992).…”

Section: Discussionmentioning

confidence: 99%

Perceived benefits of word prediction intervention on written productivity in children with spina bifida and hydrocephalus

Tam

Reid

Naumann³

et al. 2002

Occupation Therapy Intl

View full text Add to dashboard Cite

Word prediction has been commonly used as a tool to enhance written productivity. However, the effectiveness of word prediction as a strategy to meet this targeted outcome has not been established. Using a single-subject alternating treatments design, this study evaluated the effect of word prediction on written productivity from the users' perspectives. Three girls and one boy aged 10-12 with spina bifida and hydrocephalus participated in the study over a period of 20 days. The Canadian Occupational Performance Measure (COPM) was used to measure changes in perception of written productivity. Analysis of individual participant data showed that participants perceived word prediction to have the potential to influence written productivity on some writing tasks. Quantitative analysis using a randomization test did not reveal any significant changes in COPM scores after using word prediction. The varied abilities of the participants in the study and the small sample size may be the reasons why statistical analysis did not show any changes. The limitations of this study included use of a copy task, lack of a supporting measure to COPM and limited generalizability. Further studies with a larger sample are necessary to explore the skills required for successful use of word prediction and the impact of word prediction on specific tasks.

show abstract

Section: Discussionmentioning

confidence: 99%

Perceived benefits of word prediction intervention on written productivity in children with spina bifida and hydrocephalus

Tam

Reid

Naumann³

et al. 2002

Occupation Therapy Intl

View full text Add to dashboard Cite

show abstract

“…For example, DeProspero and Cohen (1979) found an interrater-agreement coefficient of just .61 (Pearson correlation) among individuals who reviewed articles for publication in behavioral journals. Other studies have found similar or even lower levels of interrater agreement among less experienced visual inspectors (e.g., Boykin & Nelson, 1981;Harbst, Ottenbacher, & Harris, 1991;Ottenbacher, 1990).…”

mentioning

confidence: 82%

Visual Aids and Structured Criteria for Improving Visual Inspection and Interpretation of Single‐case Designs

Fisher

Kelley²,

Lomas

2003

J of App Behav Analysis

289

354

View full text Add to dashboard Cite

Because behavior analysis is a data-driven process, a critical skill for behavior analysts is accurate visual inspection and interpretation of single-case data. Study 1 was a basic study in which we increased the accuracy of visual inspection methods for A-B designs through two refinements of the split-middle (SM) method called the dual-criteria (DC) and conservative dual-criteria (CDC) methods. The accuracy of these visual inspection methods was compared with one another and with two statistical methods (Allison & Gorman, 1993;Gottman, 1981) using a computer-simulated Monte Carlo study. Results indicated that the DC and CDC methods controlled Type I error rates much better than the SM method and had considerably higher power (to detect real treatment effects) than the two statistical methods. In Study 2, brief verbal and written instructions with modeling were used to train 5 staff members to use the DC method, and in Study 3, these training methods were incorporated into a slide presentation and were used to rapidly (i.e., 15 min) train a large group of individuals (N ϭ 87). Interpretation accuracy increased from a baseline mean of 55% to a treatment mean of 94% in Study 2 and from a baseline mean of 71% to a treatment mean of 95% in Study 3. Thus, Study 1 answered basic questions about the accuracy of several methods of interpreting A-B designs; Study 2 showed how that information could be used to increase the accuracy of human visual inspectors; and Study 3 showed how the training procedures from Study 2 could be modified into a format that would facilitate rapid training of large groups of individuals to interpret single-case designs.DESCRIPTORS: assessment, behavior analysis, data analysis, interrater agreement, visual inspection One area of behavioral research that continues to be a critical component of applied behavior analysis is staff training

show abstract

“…38,39 For this reason and because small treatment effects were expected, 37 we also used the C statistic to further analyze the data. 40,41 Nourbakhsh and Ottenbacher 42 used 3 statistical methods for single-subject data-the split-middle method of trend estimation, the two-standard deviation bandwidth method, and the C statistic-to analyze the same 42 graphs.…”

Section: Discussionmentioning

confidence: 99%

Effect of Passive Range of Motion Exercises on Lower-Extremity Goniometric Measurements of Adults With Cerebral Palsy: A Single-Subject Design

Cadenhead¹,

McEwen

Thompson

2002

Physical Therapy

View full text Add to dashboard Cite

Background and Purpose. People with spastic cerebral palsy often receive passive stretching that is intended to maintain or increase joint passive range of motion (PROM) even though the effectiveness of these exercises has not been definitively demonstrated. The purpose of this study was to determine the effect of PROM exercises on 6 adults with spastic quadriplegia and contractures. Participants. Four men and 2 women (Xϭ31 years of age, rangeϭ20 -44 years) who lived in an institution for people with mental retardation participated in the study. Methods. The authors used 2 multiple baseline designs. Three participants (group 1) received lower-extremity PROM exercises during phase A; PROM exercises were discontinued during phase B. Three participants (group 2) did not receive PROM exercises during phase A; PROM exercises were initiated during phase B. Data were analyzed using visual analysis and the C statistic. Results. Results varied with the method of analysis; however, phase A and phase B measurements, overall, did not differ for either group. Discussion and Conclusion. This study demonstrated use of a single-subject design to measure the effect of PROM exercises on adults with cerebral palsy. The authors concluded that the PROM exercise protocol did not have an effect on the lower-extremity goniometric measurements of the participants.[Cadenhead SL, McEwen IR, Thompson DM. Effect of passive range of motion exercises on lower-extremity goniometric measurements of adults with cerebral palsy: a single-subject design. Phys Ther. 2002;82:658 -669.]

show abstract

Interrater Reliability of Therapists' Judgments of Graphed Data

Cited by 50 publications

References 13 publications

Perceived benefits of word prediction intervention on written productivity in children with spina bifida and hydrocephalus

Perceived benefits of word prediction intervention on written productivity in children with spina bifida and hydrocephalus

Visual Aids and Structured Criteria for Improving Visual Inspection and Interpretation of Single‐case Designs

Effect of Passive Range of Motion Exercises on Lower-Extremity Goniometric Measurements of Adults With Cerebral Palsy: A Single-Subject Design

Contact Info

Product

Resources

About