Clinical Prompt Learning with Frozen Language Models

Taylor, Niall; Zhang, Yi; Joyce, Dan W.; Nevado‐Holgado, Alejo J.; Kormilitzin, Andrey

doi:10.48550/arxiv.2205.05535

Cited by 2 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…High IAA scores signify that the annotation schema is well-defined, and the annotators have a common understanding, thus contributing to the robustness and reliability of the information extraction model developed using the annotated data. The proposed information extraction (IE) and structuring pipeline will re-use and further extend our previously developed models , Senior, M. et al 2020 as well as will make use of recent advances in clinical Large Language Models with parameter-efficient fine-tuning (Taylor, N. et al 2022).…”

Section: Discussionmentioning

confidence: 99%

Identifying what anxiety treatments are provided to neurodivergent children and young people and their outcomes in routine care: protocol for a data-driven electronic health records study (ACECAT-CRIS)

Chen¹,

Kormilitzin²,

Tu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Lay SummaryAnxiety is a common problem for neurodivergent children and young people, however it can be very challenging to identify and to manage. In a national survey, many neurodivergent children and young people and their parents expressed their frustration with not getting the right support and not “being treated seriously until at a crisis point” (National Autistic Society, 2021). As a first step to understand what has been done and what else can be done to manage anxiety problems among neurodivergent children and young people, we will explore what interventions are currently offered by child and adolescent mental health services in a large NHS Trust to autistic children and young people who have problems with anxiety, and what the outcomes of these interventions are (if recorded). We will search the structural data field in current medical records and free-text clinical notes using a Natural Language Processing (NLP) model.AbstractBackground: Anxiety problems are more prominent in neurodivergent children and young people than in their neurotypical peers. Such high rates of comorbidity are important to recognise as this comorbidity is associated with a greater risk of adverse outcomes, including self-harm and suicidality, than either condition on its own. While both pharmacological and psychological treatments are often prescribed for anxiety problems in autistic children and young people, as well as in those with ADHD, the evidence base for their effectiveness has so far been inconclusive, and little is known about their treatment outcomes in routine care. Methods: This study aims to explore the nature and outcomes of anxiety treatments for autistic children and young people and for those with ADHD using pseudonymised electronic health record data. Longitudinal trends of both structured records of diagnoses (as recorded by ICD-10 codes), demographic information, treatment team and free-text clinical notes on presenting problems, indication of treatments, treatment provision, treatment outcomes and side effects, will be extracted and analysed. Free-text clinical notes will be processed to extract key concepts of interest using Natural Language Processing.Discussion: This study will explore what treatments are available for autistic children and young people, and those with ADHD, in routine care, as well as treatment outcomes. However, due to the nature and availability of data, the findings will inevitably be limited in their generalisability, which we aim to address in future phases of research.

show abstract

Section: Discussionmentioning

confidence: 99%

Identifying what anxiety treatments are provided to neurodivergent children and young people and their outcomes in routine care: protocol for a data-driven electronic health records study (ACECAT-CRIS)

Chen¹,

Kormilitzin²,

Tu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…By considering NLP problems as cloze-style tasks, prompt-based learning [30][31][32][33] has shown competitive performance in text classification tasks, especially in few-shot learning scenarios. So far, limited prior researches have studied its effectiveness on medical applications [34,35]. In this paper, we first reformulate the AD versus non-AD classification problem into probabilistically predicting label words to fill in the prompt phrases, for example, based on the template "The diagnosis is <MASK>" , where "dementia" and "healthy" serve as possible labels for "<MASK>".…”

Section: Introductionmentioning

confidence: 99%

“…The main contributions of this paper are summarized below: 1) it presents the first work adopting prompt learning based PLM finetuning for automatic AD detection. In contrast, prior researches on prompt learning for medical application were not conducted for AD detection but for clinical tasks with different characteristics [34,35].…”

Section: Introductionmentioning

confidence: 99%

Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection

Wang¹,

Deng²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care and to delay further progression. Speech based automatic AD screening systems provide a non-intrusive and more scalable alternative to other clinical screening techniques. Textual embedding features produced by pre-trained language models (PLMs) such as BERT are widely used in such systems. However, PLM domain fine-tuning is commonly based on the masked word or sentence prediction costs that are inconsistent with the back-end AD detection task. To this end, this paper investigates the use of prompt-based fine-tuning of PLMs that consistently uses AD classification errors as the training objective function. Disfluency features based on hesitation or pause filler token frequencies are further incorporated into prompt phrases during PLM fine-tuning. The exploit of the complementarity between BERT or RoBERTa based PLMs that are either prompt learning fine-tuned, or optimized using the conventional masked word or sentence prediction costs, decision voting based system combination between them is further applied. Mean, standard deviation (std) and the maximum among accuracy scores over 15 experiment runs are adopted as performance measurements for the AD detection system. Mean detection accuracy of 84.20% (with std 2.09%, best 87.5%) and 82.64% (with std 4.0%, best 89.58%) were obtained using manual and ASR speech transcripts respectively on the ADReSS20 test set consisting of 48 elderly speakers.

show abstract

Clinical Prompt Learning with Frozen Language Models

Cited by 2 publications

References 20 publications

Identifying what anxiety treatments are provided to neurodivergent children and young people and their outcomes in routine care: protocol for a data-driven electronic health records study (ACECAT-CRIS)

Identifying what anxiety treatments are provided to neurodivergent children and young people and their outcomes in routine care: protocol for a data-driven electronic health records study (ACECAT-CRIS)

Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection

Contact Info

Product

Resources

About