Siamese Neural Networks for Small Dataset Classification of Electrograms

Hunt, Bram; Kwan, Eugene; Dosdall, Derek J.; MacLeod, Rob S.; Ranjan, Ravi

doi:10.23919/cinc53138.2021.9662707

Cited by 7 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li et al [ 13 ] used SNNs for the classification of high-dimensional radiomic features extracted from MRI images. Hunt et al [ 14 ] applied SNNs for the classification of electrograms. Zhao et al [ 15 ] have used SNNs for hyperspectral image classification.…”

Section: Introductionmentioning

confidence: 99%

Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks: Algorithm Development and Validation Study

Oniani¹,

Chandrasekar²,

Sivarajkumar³

et al. 2023

JMIR AI

View full text Add to dashboard Cite

Background Natural language processing (NLP) has become an emerging technology in health care that leverages a large amount of free-text data in electronic health records to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models often requires large, annotated data sets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated data sets is typical in clinical NLP; therefore, ensuring that deep learning models perform well is crucial for real-world clinical NLP applications. A widely adopted approach is fine-tuning existing pretrained language models, but these attempts fall short when the training data set contains only a few annotated samples. Few-shot learning (FSL) has recently been investigated to tackle this problem. Siamese neural network (SNN) has been widely used as an FSL approach in computer vision but has not been studied well in NLP. Furthermore, the literature on its applications in clinical domains is scarce. Objective The aim of our study is to propose and evaluate SNN-based approaches for few-shot clinical NLP tasks. Methods We propose 2 SNN-based FSL approaches, including pretrained SNN and SNN with second-order embeddings. We evaluate the proposed approaches on the clinical sentence classification task. We experiment with 3 few-shot settings, including 4-shot, 8-shot, and 16-shot learning. The clinical NLP task is benchmarked using the following 4 pretrained language models: bidirectional encoder representations from transformers (BERT), BERT for biomedical text mining (BioBERT), BioBERT trained on clinical notes (BioClinicalBERT), and generative pretrained transformer 2 (GPT-2). We also present a performance comparison between SNN-based approaches and the prompt-based GPT-2 approach. Results In 4-shot sentence classification tasks, GPT-2 had the highest precision (0.63), but its recall (0.38) and F score (0.42) were lower than those of BioBERT-based pretrained SNN (0.45 and 0.46, respectively). In both 8-shot and 16-shot settings, SNN-based approaches outperformed GPT-2 in all 3 metrics of precision, recall, and F score. Conclusions The experimental results verified the effectiveness of the proposed SNN approaches for few-shot clinical NLP tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks: Algorithm Development and Validation Study

Oniani¹,

Chandrasekar²,

Sivarajkumar³

et al. 2023

JMIR AI

View full text Add to dashboard Cite

show abstract

“…Siamese neural networks constructed as paired twins within a shared architecture, these networks excel in capturing intricate data representations and discerning nuanced dissimilarities, rendering them particularly adept at tasks involving modest dataset sizes [2].…”

Section: Introductionmentioning

confidence: 99%

Efficiency of Supplementary Outputs in Siamese Neural Networks

Melnychenko,

Zdor

2023

A.I.S.

View full text Add to dashboard Cite

In the world of image analysis, effectively handling large image datasets is a complex challenge that requires using deep neural networks. Siamese neural networks, known for their twin-like structure, offer an effective solution to image comparison tasks, especially when data volume is limited. This research explores the possibility of enhancing these models by adding supplementary outputs that improve classification and help find specific data features. The article shows the results of two experiments using the Fashion MNIST and PlantVillage datasets, incorporating additional classification, regression, and combined output strategies with various weight loss configurations. The results from the experiments show that for simpler datasets, the introduction of supplementary outputs leads to a decrease in model accuracy. Conversely, for more complex datasets, optimal accuracy was achieved through the simultaneous integration of regression and classification supplementary outputs. It should be noted that the observed increase in accuracy is relatively marginal and does not guarantee a substantial impact on the overall accuracy of the model.

show abstract