Comparing effect sizes in follow-up studies: ROC Area, Cohen's d, and r.

Rice, Marnie E.; Harris, Grant T.

doi:10.1007/s10979-005-6832-7

Cited by 1,445 publications

(1,391 citation statements)

References 14 publications

Supporting

Mentioning

1,348

Contrasting

Unclassified

Order By: Relevance

“…AUCs for Risk Total Scores reached the large range (i.e., > .71; Rice & Harris, 2005), whereas AUCs for subscale scores generally fell in the moderate range (i.e., > .64). Cliff's delta generally fell in the medium range for both subscale and total scores (i.e., .33 -.47; Romano, Kromrey, Coraggio, Skowronek, & Devine., 2006).…”

Section: General Predictive Validitymentioning

confidence: 95%

Does reassessment of risk improve predictions? A framework and examination of the SAVRY and YLS/CMI.

Viljoen

Gray

Shaffer

et al. 2017

Psychological Assessment

View full text Add to dashboard Cite

Although experts recommend regularly reassessing adolescents' risk for violence, it is unclear whether reassessment improves predictions. Thus, in this prospective study, the authors tested 3 hypotheses as to why reassessment might improve predictions, namely the shelf-life, dynamic change, and familiarity hypotheses. Research assistants (RAs) rated youth on the Structured Assessment of Violence Risk in Youth (SAVRY) and the Youth Level of Service/Case Management Inventory (YLS/CMI) every 3 months over a 1-year period, conducting 624 risk assessments with 156 youth on probation. The authors then examined charges for violence and any offense over a 2-year follow-up period, and youths' self-reports of reoffending. Contrary to the shelf-life hypothesis, predictions did not decline or expire over time. Instead, time-dependent area under the curve scores remained consistent across the follow-up period. Contrary to the dynamic change hypothesis, changes in youth's risk total scores, compared to what is average for that youth, did not predict changes in reoffending. Finally, contrary to the familiarity hypothesis, reassessments were no more predictive than initial assessments, despite RAs' increased familiarity with youth. Before drawing conclusions, researchers should evaluate the extent to which youth receiving the usual probation services show meaningful short-term changes in risk and, if so, whether risk assessment tools are sensitive to these changes. Public Significance StatementIn this study, two adolescent risk assessment tools predicted reoffending among adolescent offenders. However, despite recommendations to reassess risk regularly, short-term reassessments did not improve predictions, suggesting a need for further research.

show abstract

Section: General Predictive Validitymentioning

confidence: 95%

Does reassessment of risk improve predictions? A framework and examination of the SAVRY and YLS/CMI.

Viljoen

Gray

Shaffer

et al. 2017

Psychological Assessment

View full text Add to dashboard Cite

show abstract

“…For the discussion, we adopted moderate to high correlations (r > .40) as minimum score. To assess the effect size of group and gender effect was used Cohen's d calculated through the free software (Devilly, 2005); small, median, and large effects are respectively correspondent to effect sizes of .20, .50, and .80 (Rice & Harris, 2005). Finally, considering that working memory could be a possible factor explaining the results, using the same withinand between-factors of first and second sets we carried out exploratory analyses with Memory of Digits as covariant, and the results were similar to the analyses reported below in that group was the covariant.…”

Section: Statistical Analysesmentioning

confidence: 99%

Number Processing and Calculation in Brazilian Children Aged 7-12 Years

et al. 2012

View full text Add to dashboard Cite

Numerical cognition is based on two components - number processing and calculation. Its development is influenced by biological, cognitive, educational, and cultural factors. The objectives of the present study were to: i) assess number processing and calculation in Brazilian children aged 7-12 years from public schools using the Zareki-R (Battery of neuropsychological tests for number processing and calculation in children, Revised; von Aster & Dellatolas, 2006) in order to obtain normative data for Portuguese speakers; ii) identify how environment, age, and gender influences the development of these mathematical skills; iii) investigate the construct validity of the Zareki-R by the contrast with the Arithmetic subtest of WISC-III. The sample included 172 children, both genders, divided in two groups: urban (N= 119) and rural (N= 53) assessed by the Zareki-R. Rural children presented lower scores in one aspect of number processing; children aged 7-8 years demonstrated an inferior global score than older; boys presented a superior performance in both number processing and calculation. Construct validity of Zareki-R was demonstrated by high to moderate correlations with Arithmetic subtest of WISC-III. The Zareki-R therefore is a suitable instrument to assess the development of mathematical skills, which is influenced by factors such as environment, age, and gender.

show abstract

“…[.54, .69], r pb = .13, p = .007) for the total, IMP, NAR, and CU scales, respectively. Overall, AUC values of total and subscale scores ranged from moderate to large (Rice & Harris, 2005).…”

Section: Predictive Validity Of the Apsdmentioning

confidence: 99%

“…One advantage of ROC analysis is that it is less sensitive to base-rate levels relative to other statistics, such as correlations, meaning that it is appropriate for relatively low base events such as offending (Rice & Harris, 2005). The AUC of the ROC graph can be taken as an index for interpreting the overall predictive accuracy of an instrument (i.e., the probability that a randomly selected youth who offended at the 6-month follow-up will have a higher APSD score than a randomly selected youth who did not offend at the 6-month follow-up).…”

Section: Predictive Validity Of the Apsdmentioning

confidence: 99%