Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts

Gupta, Shrey; Agarwal, Anmol; Roy, Kaushik; Narayanan, Vignesh; Kumaraguru, Ponnurangam; Sheth, Amit P.

doi:10.48550/arxiv.2205.13884

Cited by 2 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Language models have been employed for various text generation tasks [64], including question generation (QG) [50], [83]. QG models have enabled researchers in many fields to automatically generate their own synthetic QA datasets [84]- [87]. Our dataset was partially generated using QG.…”

Section: Related Workmentioning

confidence: 99%

AI-based Question Answering Assistance for Analyzing Natural-language Requirements

Ezzini¹,

Abualhaija²,

Arora³

et al. 2023

Preprint

View full text Add to dashboard Cite

By virtue of being prevalently written in natural language (NL), requirements are prone to various defects, e.g., inconsistency and incompleteness. As such, requirements are frequently subject to quality assurance processes. These processes, when carried out entirely manually, are tedious and may further overlook important quality issues due to time and budget pressures. In this paper, we propose QAssist -a questionanswering (QA) approach that provides automated assistance to stakeholders, including requirements engineers, during the analysis of NL requirements. Posing a question and getting an instant answer is beneficial in various quality-assurance scenarios, e.g., incompleteness detection. Answering requirementsrelated questions automatically is challenging since the scope of the search for answers can go beyond the given requirements specification. To that end, QAssist provides support for mining external domain-knowledge resources. Our work is one of the first initiatives to bring together QA and external domain knowledge for addressing requirements engineering challenges. We evaluate QAssist on a dataset covering three application domains and containing a total of 387 question-answer pairs. We experiment with state-of-the-art QA methods, based primarily on recent large-scale language models. In our empirical study, QAssist localizes the answer to a question to three passages within the requirements specification and within the external domainknowledge resource with an average recall of 90.1% and 96.5%, respectively. QAssist extracts the actual answer to the posed question with an average accuracy of 84.2%.

show abstract

Section: Related Workmentioning

confidence: 99%

AI-based Question Answering Assistance for Analyzing Natural-language Requirements

Ezzini¹,

Abualhaija²,

Arora³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Mental health disorders such as Major Depressive Disorder (MDD) 1 and Anxiety Disorder (AD) 2 are widespread; 20.6% and 4.3% in the USA before the pandemic 3 . The current pandemic has further aggravated this issue.…”

Section: Introductionmentioning

confidence: 99%

“…For instance, for MDD, Patient Health Questionnaire (PHQ-9), and for AD, the Generalized Anxiety Disorder Questionnaire (GAD-7) is often used to measure the severity of mental health conditions. These questionnaires are what we consider process knowledge (ProKnow) [1,2,3,4]. Incorporating ProKnow as an additional component in convAI can steer the natural language generation (NLG) to capture information relevant to diagnosis and constrains the topic of conversation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ProKnow: Process knowledge for safety constrained and explainable question generation for mental health diagnostic assistance

Roy¹,

Soltani²,

Rawte³

et al. 2023

Front. Big Data

View full text Add to dashboard Cite

Virtual Mental Health Assistants (VMHAs) are utilized in health care to provide patient services such as counseling and suggestive care. They are not used for patient diagnostic assistance because they cannot adhere to safety constraints and specialized clinical process knowledge (ProKnow) used to obtain clinical diagnoses. In this work, we define ProKnow as an ordered set of information that maps to evidence-based guidelines or categories of conceptual understanding to experts in a domain. We also introduce a new dataset of diagnostic conversations guided by safety constraints and ProKnow that healthcare professionals use (ProKnow-data). We develop a method for natural language question generation (NLG) that collects diagnostic information from the patient interactively (ProKnow-algo). We demonstrate the limitations of using state-of-the-art large-scale language models (LMs) on this dataset. ProKnow-algo incorporates the process knowledge through explicitly modeling safety, knowledge capture, and explainability. As computational metrics for evaluation do not directly translate to clinical settings, we involve expert clinicians in designing evaluation metrics that test four properties: safety, logical coherence, and knowledge capture for explainability while minimizing the standard cross entropy loss to preserve distribution semantics-based similarity to the ground truth. LMs with ProKnow-algo generated 89% safer questions in the depression and anxiety domain (tested property: safety). Further, without ProKnow-algo generations question did not adhere to clinical process knowledge in ProKnow-data (tested property: knowledge capture). In comparison, ProKnow-algo-based generations yield a 96% reduction in our metrics to measure knowledge capture. The explainability of the generated question is assessed by computing similarity with concepts in depression and anxiety knowledge bases. Overall, irrespective of the type of LMs, ProKnow-algo achieved an averaged 82% improvement over simple pre-trained LMs on safety, explainability, and process-guided question generation. For reproducibility, we will make ProKnow-data and the code repository of ProKnow-algo publicly available upon acceptance.

show abstract

Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts

Cited by 2 publications

References 19 publications

AI-based Question Answering Assistance for Analyzing Natural-language Requirements

AI-based Question Answering Assistance for Analyzing Natural-language Requirements

ProKnow: Process knowledge for safety constrained and explainable question generation for mental health diagnostic assistance

Contact Info

Product

Resources

About