EDNet: Attention-Based Multimodal Representation for Classification of Twitter Users Related to Eating Disorders

Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing Approaches

“…These opinions were sourced from trained psychiatrists or psychologists [ 112 , 131 , 145 , 186 , 219 ], as well as staff and students within the university settings with backgrounds in psychology [ 142 , 185 ]. For example, Abuhassan et al [ 218 ] incorporated opinions from domain experts with specific expertise in eating disorders (EDs), psychology, mental health, and social media. The authors obtained a comprehensive and well-rounded annotation strategy to guide the categorization of social media users into individuals with an explicit diagnosis of EDs, healthcare professionals, communicators (i.e., those who communicate, exchange, and distribute information to the public), and non-ED individuals.…”

“… Features Tools Studies Feature Category Count of words: general, condition-specific (depressed, suicidal, eating disorder-related) keywords, emojis N/A [ 20 , 104 , 109 , 123 , 126 , 127 , 130 , 133 , 134 , 137 , 145 , 146 , 187 , 188 , 218 , 219 ] Linguistic Words referring to social processes (e.g., reference to family, friends, social affiliation), and psychological states (e.g., negative/positive emotions) Linguistic Inquiry and Word Count (LIWC) [ 278 ], LIWC 2007 Spanish dictionary [ 369 ], Chinese Suicide Dictionary [ 370 ], Chinese LIWC [ 371 ], TextMind [ 372 ], Suite of Automatic Linguistic Analysis Tools (SALAT) [ 279 ]—Simple Natural Language Processing (SiNLP) [ 373 ] [ 20 , 79 , 109 , 118 , 121 , 128 , 186 , 194 , 196 , ...…”

“…Words referring to social processes (e.g., reference to family, friends, social affiliation), and psychological states (e.g., negative/positive emotions) Linguistic Inquiry and Word Count (LIWC) [278], LIWC 2007 Spanish dictionary [369], Chinese Suicide Dictionary [370], Chinese LIWC [371], TextMind [372], Suite of Automatic Linguistic Analysis Tools (SALAT) [279]-Simple Natural Language Processing (SiNLP) [373] [ 20,79,109,118,121,128,186,194,[196][197][198]204,211,219,374] Linguistic Part-of-speech (POS) tags: adjectives, nouns, pronouns Jieba [375], Natural Language Toolkit (NLTK) [280], TextBlob [376], spaCy, Penn Treebank [377], Empath [378] [ 61,100,104,123,126,135,184,185,189,195,218,219] Linguistic Table A3. Cont.…”

“…Learn shared representations from weighted modality-specific representations Gated Multimodal Unit (GMU) [429], parallel attention model, attention layer, sparse MLP (mix vertical and horizontal information via weight sharing and sparse connection), multimodal encoder-decoder, multimodal factorized bilinear pooling (combines compact output features of multi-modal low-rank bilinear [430] and robustness of multi-modal compact bilinear [431]), multi-head intermodal attention fusion, transformer [295], feed-forward network, low-rank multimodal fusion network [432] [ 62,65,67,76,93,100,102,106,113,117,131,135,136,[142][143][144]174,218,433] Learn joint sparse representations Dictionary learning [20] Learn and fuse outputs from different modality-specific parts at fixed time steps Cell-coupled LSTM with L-skip fusion mechanism [101] Learn cross-modality representations that incorporate interactions between modalities LXMERT [434], transformer encoder with cross-attention layers (representations of a modality as query and the other as key/value, and vice versa), memory fusion network [435] [82, 92,129] Horizontal and vertical kernels to capture patterns across different levels CASER [309] [170]…”

“…Fully connected (FC) layers, multilayer perceptron (MLP), CNN, LSTM, BiLSTM, GRU, temporal convolutional network (TCN) [441] (with dilation for long sequences)-with activation function like Sigmoid, Softmax, ReLU, LeakyReLU, and GeLU Predict scores of assessment scales (regression) or probability distribution over classes (classification) [60,78,80,[84][85][86][87][88][90][91][92][93][94]96,98,105,111,113,117,131,133,135,136,[142][143][144]146,162,163,167,168,170,172,174,178,179,190,197,199,201,218,219,221,223,308] DCNN-DNN (combination of deep CNN and DNN), GCNN-LSTM (combination of gated convolutional neural network, which replaces a convolution block in CNN wi...…”