Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks Track 2

Filannino, Michele; Stubbs, Amber; Uzuner, Özlem

doi:10.1016/j.jbi.2017.04.017

Cited by 38 publications

(53 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most misclassifications were between mild/moderate and moderate/severe classes. Since annotators of expert clinicians also disagreed most among these classes[2], it seems hard to distinguish them solely based on psychiatric notes. Also, our system made the most errors within the moderate class, which happens to be the one that varied the most between the training and test set with respect to the frequency distribution among all classes.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Note that this measure gives the same importance to every class, regardless of its relative frequency. [2]…”

Section: Methodsmentioning

confidence: 99%

“…Track 2 of the 2016 CEGS N-GRID Shared Task in the Clinical Natural Language focuses on one domain: positive valence. [2] The organizers provided a corpus composed of initial psychiatric evaluation records of patients along with a general severity score (0–3) of positive valence domain for each note annotated by expert clinicians. The task was to build a system that can automatically predict an overall positive valence severity score based on the patients’ psychiatric notes.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Symptom severity classification with gradient tree boosting

Liu

Nguyen

et al. 2017

Journal of Biomedical Informatics

View full text Add to dashboard Cite

In this paper, we present our system as submitted in the CEGS N-GRID 2016 task 2 RDoC classification competition. The task was to determine symptom severity (0–3) in a domain for a patient based on the text provided in his/her initial psychiatric evaluation. We first preprocessed the psychiatry notes into a semi-structured questionnaire and transformed the short answers into either numerical, binary, or categorical features. We further trained weak Support Vector Regressors (SVR) for each verbose answer and combined regressors’ output with other features to feed into the final gradient tree boosting classifier with resampling of individual notes. Our best submission achieved a macro-averaged Mean Absolute Error of 0.439, which translates to a normalized score of 81.75%.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

“…Note that this measure gives the same importance to every class, regardless of its relative frequency. [2]…”

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Symptom severity classification with gradient tree boosting

Liu

Nguyen

et al. 2017

Journal of Biomedical Informatics

View full text Add to dashboard Cite

show abstract

“…We evaluated our approach using the official results provided by the organizers of the CEGS/N-GRID workshop [29]. A total of 65 runs were submitted by 24 teams.…”

Section: Resultsmentioning

confidence: 99%

“…In 2016 the Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-Scale and RDoC Individualized Domains (N-GRID) organized a new challenge which aimed to extract symptom severity from neuropsychiatric clinical records [10]. The RDoC for Psychiatry Challenge of the CEGS/N-GRID focused on just one domain of functioning, namely: positive valence .…”

Section: Task Descriptionmentioning

confidence: 99%

Automatic recognition of symptom severity from psychiatric evaluation records

Goodwin

Maldonado

Harabagiu

2017

Journal of Biomedical Informatics

View full text Add to dashboard Cite

This paper presents a novel method for automatically recognizing symptom severity by using natural language processing of psychiatric evaluation records to extract features that are processed by machine learning techniques to assign a severity score to each record evaluated in the 2016 RDoC for Psychiatry Challenge from CEGS/N-GRID. The natural language processing techniques focused on (a) discerning the discourse information expressed in questions and answers; (b) identifying medical concepts that relate to mental disorders; and (c) accounting for the role of negation. The machine learning techniques rely on the assumptions that (1) the severity of a patient’s positive valence symptoms exists on a latent continuous spectrum; and (2) all the patient’s answers and narratives documented in the psychological evaluation records are informed by the patient’s latent severity score along this spectrum. These assumptions motivated our two-step machine learning framework for automatically recognizing psychological symptom severity. In the first step, the latent continuous severity score is inferred from each record; in the second step, the severity score is mapped to one of the four discrete severity levels used in the CEGS/N-GRID challenge. We evaluated three methods for inferring the latent severity score associated with each record: (i) pointwise ridge regression; (ii) pairwise comparison-based classification; and (iii) a hybrid approach combining pointwise regression and the pairwise classifier. The second step was implemented using a tree of cascading support vector machine (SVM) classifiers. While the official evaluation results indicate that all three methods are promising, the hybrid approach not only outperformed the pairwise and pointwise methods, but also produced the second highest performance of all submissions to the CEGS/N-GRID challenge with a normalized MAE score of 84.093% (where higher numbers indicate better performance). These evaluation results enabled us to observe that, for this task, considering pairwise information can produce more accurate severity scores than pointwise regression – an approach widely used in other systems for assigning severity scores. Moreover, our analysis indicates that using a cascading SVM tree outperforms traditional SVM classification methods for the purpose of determining discrete severity levels.

show abstract

Research Domain Criteria scores estimated through natural language processing are associated with risk for suicide and accidental death

2019

View full text Add to dashboard Cite

Background: Identification of individuals at increased risk for suicide is an important public health priority, but the extent to which considering clinical phenomenology improves prediction of longer term outcomes remains understudied. Hospital discharge provides an opportunity to stratify risk using readily available clinical records and details. Methods:We applied a validated natural language processing tool to generate estimated Research Domain Criteria (RDoC) scores for a cohort of 444,317 individuals drawn from 815,457 hospital discharges between 2005 and 2013. We used survival analysis to examine the association of this risk with suicide and accidental death, adjusted for sociodemographic features. Results:In adjusted models, symptoms in each of the five domains contributed to incremental risk (log rank P < 0.001), with greatest increase observed with positive valence. The contribution of each domain to risk was time dependent.Conclusions: RDoC symptom scores parsed from clinical documentation are associated with suicide and illustrates that multiple domains contribute to risk in a time-varying fashion.

show abstract

Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks Track 2

Cited by 38 publications

References 20 publications

Symptom severity classification with gradient tree boosting

Symptom severity classification with gradient tree boosting

Automatic recognition of symptom severity from psychiatric evaluation records

Research Domain Criteria scores estimated through natural language processing are associated with risk for suicide and accidental death

Contact Info

Product

Resources

About