Background and Hypothesis Despite decades of “proof of concept” findings supporting the use of Natural Language Processing (NLP) in psychosis research, clinical implementation has been slow. One obstacle reflects the lack of comprehensive psychometric evaluation of these measures. There is overwhelming evidence that criterion and content validity can be achieved for many purposes, particularly using machine learning procedures. However, there has been very little evaluation of test-retest reliability, divergent validity (sufficient to address concerns of a “generalized deficit”), and potential biases from demographics and other individual differences. Study Design This article highlights these concerns in development of an NLP measure for tracking clinically rated paranoia from video “selfies” recorded from smartphone devices. Patients with schizophrenia or bipolar disorder were recruited and tracked over a week-long epoch. A small NLP-based feature set from 499 language samples were modeled on clinically rated paranoia using regularized regression. Study Results While test–retest reliability was high, criterion, and convergent/divergent validity were only achieved when considering moderating variables, notably whether a patient was away from home, around strangers, or alone at the time of the recording. Moreover, there were systematic racial and sex biases in the model, in part, reflecting whether patients submitted videos when they were away from home, around strangers, or alone. Conclusions Advancing NLP measures for psychosis will require deliberate consideration of test-retest reliability, divergent validity, systematic biases and the potential role of moderators. In our example, a comprehensive psychometric evaluation revealed clear strengths and weaknesses that can be systematically addressed in future research.

Social distancing policies enacted during the COVID-19 pandemic altered our social interactions. People with schizophrenia, who already exhibit social deficits, may have been disproportionally impacted. In this pilot study, we a) compared prepandemic social functioning to functioning during the pandemic in people with schizophrenia (n = 21) who had data at both time points; and b) examined if patterns of decline in schizophrenia differed from healthy controls (n = 21) across a series of repeated-measures analyses of variance. We observed larger declines in social functioning in schizophrenia (η2 = 0.07, medium effect size) during the pandemic compared with the control group. Between-group declines did not extend to other domains, suggesting that declines are specific to social functioning. Our findings signal that treatments focusing on reconnecting people with schizophrenia to their social networks should be prioritized. Future studies should continue tracking social functioning after the pandemic to illustrate patterns of recovery.