Multi-input Recurrent Independent Mechanisms for leveraging knowledge sources: Case studies on sentiment analysis and health text mining

Abstract

This paper presents a way to inject and leverage existing knowledge from external sources in a Deep Learning environment, extending the recently proposed Recurrent Independent Mechnisms (RIMs) architecture, which comprises a set of interacting yet independent modules. We show that this extension of the RIMs architecture is an effective framework with lower parameter implications compared to purely fine-tuned systems.

Automatic Extraction of Medication Mentions from Tweets—Overview of the BioCreative VII Shared Task 3 Competition

“…Moreover, morphological information is encoded using a character-based CNN ( 51 ). The external information is leveraged together with ClinicalBERT embeddings ( 35 ) using the multi-input Recurrent Independent Mechanisms model ( 52 ), where each module is responsible for incorporating one information source. In addition to external resources, after the first training epoch, a Blacklist (FP terms) and a Whitelist (FN terms) are automatically generated by evaluating the model on the development set.…”

Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI)

“…The recently proposed multi-input RIM framework (Bagherzadeh and Bergler, 2021) comes close to our ideas and we use it here for decoupled integration of our KSs. (Bagherzadeh and Bergler, 2021) showed successful decoupled integration of simple KSs like gazetteer lists that were task appropriate but did not report on experiments with large, structured KSs.…”

“…As described in (Bagherzadeh and Bergler, 2021), mi-RIM is an architecture of M decoupled recurrent modules f 1 , . .…”

“…Because the input selection and interaction mechanisms are attention based and attention can take a variable number of argument representation, new KS modules can be added to an existing model without major changes. (Bagherzadeh and Bergler, 2021) provided a proof of concept for integration of language models and a few gazetteer lists on simple tweetrelated biomedical tasks. Here, instead, we test the decoupled mi-RIM framework on complex tasks and on a more diverse set of KSs.…”