The Social Shapes Test as a Self-Administered, Online Measure of Social Intelligence: Two Studies with Typically Developing Adults and Adults with Autism Spectrum Disorder

Brown, Matt I.; Heck, Patrick R.; Chabris, Christopher F.

doi:10.1007/s10803-023-05901-2

Cited by 3 publications

(4 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In both SJTs, all items were presented on one page. Test instructions were shown on a separate page before the SJT as well as on top of the page with the actual test items (for the instruction we used for the SST, see [37]). In the SST, participants had the option to view the videos as many times as they liked.…”

Section: Sample and Proceduresmentioning

confidence: 99%

“…Finally, the highly significant, albeit rather weak, correlation (r = 0.26, p = < 0.001) between the SST and the HAM-SJT (see Table 2) indicates that the SST measures similar aspects to the HAM-SJT. However, test developers of the SST could already show promising convergent validity with some SJTs and other performance measures of social competence [28,37]. For its use in the medical school admission context, however, more studies are required first.…”

Section: Limitations and Future Researchmentioning

confidence: 99%

See 1 more Smart Citation

The effects of language proficiency and awareness of time limit in animated vs. text-based situational judgment tests

Rabe,

Gröne,

von Bernstorff

et al. 2024

BMC Med Educ

View full text Add to dashboard Cite

Background Situational Judgment Tests (SJTs) are commonly used in medical school admissions. However, it has been consistently found that native speakers tend to score higher on SJTs than non-native speakers, which can be particularly problematic in the admission context due to the potential risk of limited fairness. Besides type of SJT, awareness of time limit may play a role in subgroup differences in the context of cognitive load theory. This study examined the influence of SJT type and awareness of time limit against the background of language proficiency in a quasi high-stakes setting. Methods Participants (N = 875), applicants and students in healthcare-related study programs, completed an online study that involved two SJTs: one with a text-based stimulus and response format (HAM-SJT) and another with a video-animated stimulus and media-supported response format (Social Shapes Test, SST). They were randomly assigned to a test condition in which they were either informed about a time limit or not. In a multilevel model analysis, we examined the main effects and interactions of the predictors (test type, language proficiency and awareness of time limit) on test performance (overall, response percentage). Results There were significant main effects on overall test performance for language proficiency in favor of native speakers and for awareness of time limit in favor of being aware of the time limit. Furthermore, an interaction between language proficiency and test type was found, indicating that subgroup differences are smaller for the animated SJT than for the text-based SJT. No interaction effects on overall test performance were found that included awareness of time limit. Conclusion A SJT with video-animated stimuli and a media-supported response format can reduce subgroup differences in overall test performance between native and non-native speakers in a quasi high-stakes setting. Awareness of time limit is equally important for high and low performance, regardless of language proficiency or test type.

show abstract

Section: Sample and Proceduresmentioning

confidence: 99%

Section: Limitations and Future Researchmentioning

confidence: 99%

The effects of language proficiency and awareness of time limit in animated vs. text-based situational judgment tests

Rabe,

Gröne,

von Bernstorff

et al. 2024

BMC Med Educ

View full text Add to dashboard Cite

show abstract

“…Brown et al (2019) report a full account of the initial development and validity evidence for the SST among typically-developing adult samples and more recent research has demonstrated convergent validity with effective judgment in social situations in the workplace (Brown et al, 2022) along with performance on similar animated shape tasks (Brown et al, 2021). The validity of animated shape tasks has been demonstrated in clinical and developmental research, including in identifying difficulties in social attribution among individuals diagnosed with autism spectrum disorder or schizophrenia (Burger-Caplan et al, 2016;Brown, Heck, & Chabris, 2023;Johannesen et al, 2018;Martinez et al, 2019;Wilson, 2021). The SST therefore shows promise as an effective measure of social intelligence in clinical and non-clinical populations.…”

Section: Objective Social Intelligencementioning

confidence: 99%

A Robust Negative Relationship Between Self-Reports of Social Skills and Performance Measures of Social Intelligence

Heck,

Brown,

Chabris

2024

Swiss Psychology Open

View full text Add to dashboard Cite

Does claiming to be more socially skilled than average reflect above average social intelligence? People may define social skill subjectively or idiosyncratically, and feedback about one's own level of social intelligence or specific social skills may be inconsistent or biased. This suggests that people's self-evaluations of their own social skills may fail to predict-or even negatively predict-their performance on skill-based tests of social intelligence. In one exploratory study and one preregistered replication study (total N = 927), people who reported that they have better social skills than average performed worse on three skill-based tests of social intelligence than people who did not claim to be above average (d Study1 = 0.46, d Study2 = 0.58). This pattern did not emerge when considering participants' comparative judgments of their own general intelligence and performance on two tests of cognitive ability. We discuss the complexities of accurately assessing one's own social skills, the obstacles associated with measuring the construct of social intelligence using self-reports, and the strategic implications of claiming to be more socially skilled than average.

show abstract

“…Data were analyzed using R, version 3.6.1 (R Core Team, 2019). The data reported in this study were collected as part of a test-retest reliability study reported in Brown et al (2023).…”

Section: Transparency and Opennessmentioning

confidence: 99%

Exploring the Role of Social Desirability in Situational Judgment Tests

Brown,

Martin-Raugh

2024

Preprint

View full text Add to dashboard Cite

We explore the role of social desirability and social norms in explaining how individuals respond to situational judgment test (SJT) items. We argue that SJTs can be conceptualized as measuring the knowledge of socially desirable behavior or norms in a situation or context. Consensus-scored SJTs consistently correlated positively with GMA, knowledge of social norms, and socially desirable traits in samples of undergraduate students and adults from an online crowdsourcing platform regardless of test content. Additionally, we found that trait-focused SJTs function differently compared to traditional, skill-based SJTs when expert-based scoring is used. Trait-focused SJTs contain response option sets which feature differing levels of trait expression, social desirability, and effectiveness. This results in greater variability in responding due to differences in context (e.g., different instructions or organizational setting) and in greater potential for multiple scoring keys with differing patterns of convergent validity. We hope that our findings help bring attention to the role of social norms and desirability when studying SJTs and encourage more emphasis on the characteristics of response options to complement prior research focused on the situational stems.

show abstract

The Social Shapes Test as a Self-Administered, Online Measure of Social Intelligence: Two Studies with Typically Developing Adults and Adults with Autism Spectrum Disorder

Cited by 3 publications

References 78 publications

The effects of language proficiency and awareness of time limit in animated vs. text-based situational judgment tests

The effects of language proficiency and awareness of time limit in animated vs. text-based situational judgment tests

A Robust Negative Relationship Between Self-Reports of Social Skills and Performance Measures of Social Intelligence

Exploring the Role of Social Desirability in Situational Judgment Tests

Contact Info

Product

Resources

About