Enhancing text comprehension for Question Answering with Contrastive Learning

Abstract

Most of the existing medication recommendation models are predicted with only structured data such as medical codes, with the remaining other large amount of unstructured or semi-structured data underutilization. To increase the utilization effectively, we proposed a method of enhancing medication recommendation with Large Language Model (LLM) text representation. LLM harnesses powerful language understanding and generation capabilities, enabling the extraction of information from complex and lengthy unstructured data such as clinical notes which contain complex terminology. This method can be applied to several existing base models we selected and improve medication recommendation performance with the combination representation of text and medical codes experiments on two different datasets.LLM text representation alone can even demonstrate a comparable ability to the medical code representation alone. Overall, this is a general method that can be applied to other models for improved recommendations.

Training Process of Unsupervised Learning Architecture for Gravity Spy Dataset

“…Furthermore, we will use our unsupervised classification in KAGRA to develop a transient noise system. Additionally, we will extend our architecture to self‐supervised learning [ 41 ] to improve classification accuracy, in which the architecture generates pseudo labels for a given dataset and re‐trains it.…”

Exploring simple triplet representation learning

“…Contrastive learning, in particular, encourages models to generate diverse augmented representations from the same input. Prior to contrastive learning, numerous works sought to enforce invariance in representations through different methodologies, such as pseudo-labelling [31] and virtual adversarial training [32] .…”