Text-to-image artificial intelligence to aid clinicians in perceiving unique neuro-ophthalmic visual phenomena

Waisberg, Ethan; Ong, Joshua; Masalkhi, Mouayad; Zaman, Nasif; Kamran, Sharif Amit; Sarker, Prithul; Lee, Andrew G.; Tavakkoli, Alireza

doi:10.1007/s11845-023-03315-8

Cited by 26 publications

(24 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LLMs are used to enable various general healthcare applications. [94] showed that GPT-4 has the potential to help drive medical innovation, from aiding with patient discharge notes, summarizing recent clinical trials, and providing information on ethical guidelines.…”

Section: Llm-based Healthcare and Mental Healthcarementioning

confidence: 99%

AI Therapist for Daily Functioning Assessment and Intervention Using Smart Home Devices

Nie

Zhao

Xia

et al. 2022

Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems

View full text Add to dashboard Cite

Fig. 1. CaiTI is a conversational "AI therapist" that screens and assesses the day-to-day functioning of the occupant across 37 dimensions and provides appropriate psychotherapeutic interventions and empathic validations depending on the physical and mental status of the occupant. Despite the global mental health crisis, access to screenings, professionals, and treatments remains high. In collaboration with licensed psychotherapists, we propose a Conversational AI Therapist with psychotherapeutic Interventions (CaiTI), a platform that leverages large language models (LLM)s and smart devices to enable better mental health self-care. CaiTI can screen the day-to-day functioning using natural and psychotherapeutic conversations. CaiTI leverages reinforcement learning to provide personalized conversation flow. CaiTI can accurately understand and interpret user responses. When the user needs further attention during the conversation, CaiTI can provide conversational psychotherapeutic interventions, including cognitive behavioral therapy (CBT) and motivational interviewing (MI). Leveraging the datasets prepared by the licensed psychotherapists, we experiment and microbenchmark various LLMs' performance in tasks along CaiTI's conversation flow and discuss their strengths and weaknesses. With the psychotherapists, we implement CaiTI and conduct 14-day and 24-week studies. The study results, validated by therapists, demonstrate that CaiTI can converse with users naturally, accurately understand and interpret user responses, and provide psychotherapeutic interventions

show abstract

Section: Llm-based Healthcare and Mental Healthcarementioning

confidence: 99%

AI Therapist for Daily Functioning Assessment and Intervention Using Smart Home Devices

Nie

Zhao

Xia

et al. 2022

Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems

View full text Add to dashboard Cite

show abstract

“…Recent advances in the DL field, mainly driven by the proposal of the transformer architecture [ 19 ], have seen breakthroughs in the analysis of different modalities, such as text with models like BERT [ 20 ] or GPT [ 21 ] or images with models like Vision Transformers [ 22 ]. These models have been a revolution also in specific fields such as medicine with impressive performances in medical applications such as medical tests or diagnosis [ 23 – 25 ]. However, some of the limitations of working with these data modalities and these specialized DL models are, the high dimensionality of the data, the heterogeneity of the diverse data modalities and formats, and the amount of data and computational resources required to train robust models for these specific modalities [ 26 , 27 ].…”

Section: Introductionmentioning

confidence: 99%

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era

Restrepo,

Wu,

Vásquez-Venegas

et al. 2024

Preprint

View full text Add to dashboard Cite

In the big data era, integrating diverse data modalities poses significant challenges, particularly in complex fields like healthcare. This paper introduces a new process model for multimodal Data Fusion for Data Mining, integrating embeddings and the Cross-Industry Standard Process for Data Mining with the existing Data Fusion Information Group model. Our model aims to decrease computational costs, complexity, and bias while improving efficiency and reliability. We also propose "disentangled dense fusion," a novel embedding fusion method designed to optimize mutual information and facilitate dense inter-modality feature interaction, thereby minimizing redundant information.We demonstrate the model's efficacy through three use cases: predicting diabetic retinopathy using retinal images and patient metadata, domestic violence prediction employing satellite imagery, internet, and census data, and identifying clinical and demographic features from radiography images and clinical notes. The model achieved a Macro F1 score of 0.92 in diabetic retinopathy prediction, an R-squared of 0.854 and sMAPE of 24.868 in domestic violence prediction, and a macro AUC of 0.92 and 0.99 for disease prediction and sex classification, respectively, in radiological analysis. These results underscore the Data Fusion for Data Mining model's potential to significantly impact multimodal data processing, promoting its adoption in diverse, resource-constrained settings.

show abstract

“…ChatGPT has already been capable of tasks such as generating artificial intelligence (AI algorithms), or generating images based on patient descriptions of complex neuro-ophthamic visual phenomena. 2 …”

Section: Introductionmentioning

confidence: 99%

ChatGPT and medical education: a new frontier for emerging physicians

Waisberg,

Ong,

Masalkhi

et al. 2023

Can. Med. Ed. J

Self Cite

View full text Add to dashboard Cite

Text-to-image artificial intelligence to aid clinicians in perceiving unique neuro-ophthalmic visual phenomena

Cited by 26 publications

References 11 publications

AI Therapist for Daily Functioning Assessment and Intervention Using Smart Home Devices

AI Therapist for Daily Functioning Assessment and Intervention Using Smart Home Devices

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era

ChatGPT and medical education: a new frontier for emerging physicians

Contact Info

Product

Resources

About