Counseling use case analysis of AI-based chatbot And Suggestion for developing a chatbot for School counseling

Lee, Jae Yong; Lee, Sihoon; Kwoon, Junghyeon; Joonsup, Choi

doi:10.28972/kjec.2022.21.3.243

Cited by 4 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The GPT, developed by OpenAI [14], is a type of AI model used for NLP tasks. Following the research path from the original GPT, GPT-2, and GPT-3, OpenAI's DL approach leverages extensive amounts of data and intensive computation to create increasingly sophisticated and capable language models [18].…”

Section: Generative Pretrained Transformermentioning

confidence: 99%

“…Recently, InstructGPT (GPT-3.5)-a GPT model employing 175 billion parameters with supervised fine-tuning and reinforcement learning from human feedback [11]-and its dialogueoptimized chatbot (ChatGPT) launched in November 2022 have significantly impacted NLP fields [13]. By predicting the subsequent element of the texts, ChatGPT can comprehend user prompts and generate human-like responses, expressed in ethical, sentimental, logical, and creative manner, without any additional training (e.g., foundation model) [14]. Although GPT is a non-domainspecific LLM, not exclusively intended to be used for medical or healthcare fields, recent publications have demonstrated that ChatGPT (GPT-3.5) possesses sufficient ability to pass the United States Medical Licensing Examination [15,16].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in Japan

Tanaka

Nakata

Aiga

et al. 2023

Preprint

View full text Add to dashboard Cite

The remarkable performance of ChatGPT, launched in November 2022, has significantly impacted the field of natural language processing, inspiring the application of large language models as supportive tools in clinical practice and research worldwide. Although ChatGPT recently scored high on the United States Medical Licensing Examination, its performance on medical licensing examinations of other nations, especially non-English speaking nations, has not been sufficiently evaluated. This study assessed ChatGPT's performance on the National Medical Licensing Examination (NMLE) in Japan and compared it with the actual minimal passing rate for this exam. In particular, the performances of both the GPT-3.5 and GPT-4 models were considered for the comparative analysis. We initially used a model and prompt tuning set of 290 questions without image data from the previous 116th NMLE (held in February 2022) to maximize the performance for delivering correct answers and explanations of the questions. Thereafter, we tested the performance of the best ChatGPT model (GPT-4) with tuned prompts on a dataset of 262 questions without images from the latest 117th NMLE (held in February 2023). The best model with the tuned prompts scored 82.7% for the essential questions and 77.2% for the basic and clinical questions, both of which sufficed the minimum passing rates of 80.0% and 74.6%, respectively. Simultaneously, we identified the three major factors contributing to the generation of the incorrect answers: insufficient medical knowledge, information on Japan-specific medical system and guidelines, and mathematical errors. In conclusion, GPT-4 powered ChatGPT with our optimally tuned prompts achieved a minimum passing rate in the latest 117th NMLE in Japan. Although we express strong concerns regarding the use of the current ChatGPT for medical purposes so far, these artificial intelligence models may soon have the potential to serve as one of the best sidekicks for solving medical and healthcare problems.

show abstract

Section: Generative Pretrained Transformermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in Japan

Tanaka

Nakata

Aiga

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition, prompts are not limited to being sentences in English. They can be written in several human languages and may include data inputs such as spreadsheets, technical specifications, research papers, and mathematical equations (3). Despite recent advancements, there is a lack of data on how pathologists currently apply these technologies to their professional lives.…”

Section: Introductionmentioning

confidence: 99%

Adoption of AI-Powered Chatbots with Large Language Models by Pathologists

Bychkov,

Laohawetwanit,

Pinto

2024

Preprint

View full text Add to dashboard Cite

Aims: The study aimed to investigate the adoption and perception of artificial intelligence (AI) chatbots, particularly those powered by large language models (LLMs), among pathologists worldwide. It explored the extent of their engagement with these technologies, identifying potential impacts on their professional practices. Methods: A cross-sectional survey was conducted, gathering data from pathologists on their usage and views concerning AI chatbots powered by LLMs. The survey, distributed globally via various digital platforms, included both quantitative and qualitative questions. Statistical analyses were performed to delineate patterns in the adoption and perspectives on these AI tools among the respondents. Results: Of 215 respondents, 100 (46.5%) reported using LLMs, particularly ChatGPT, for professional purposes, predominantly for information retrieval, proofreading, and academic writing, highlighting a significant time-saving benefit. The adoption varied across demographics, with younger, male pathologists showing higher usage rates. While the technology was mainly utilized for drafting academic materials and programming tasks, users expressed concerns about information accuracy, privacy, and the need for regulatory approval. Despite recognizing occasional inaccuracies, respondents saw potential in advanced AI features, particularly in image analysis and speech-to-text functions. Conclusions: The survey underscored pathologists' cautious yet growing interest in leveraging LLMs to enhance information accessibility, efficiency, and medical education. While the potential benefits are recognized, significant apprehensions about the reliability, ethics, and security associated with these AI tools underscore the need for comprehensive regulation and standardized practices to ensure their responsible use in the medical field.

show abstract

“…2,3 Despite controversies and unsettled questions, healthcare is poised for major near-term changes as AI is implemented into workflows. 4 One crucial issue involves understanding how AI tools perform and how "choices" made by AI compare with those made by humans, including people with different levels of expertise. [5][6][7] This can be helpful to test the accuracy of models and to examine whether confounders are present.…”

Section: Introductionmentioning

confidence: 99%

Human and computer attention in assessing genetic conditions

Duong,

Johny,

Hanchard

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep learning (DL) and other types of artificial intelligence (AI) are increasingly used in many biomedical areas, including genetics. One frequent use in medical genetics involves evaluating images of people with potential genetic conditions to help with diagnosis. A central question involves better understanding how AI classifiers assess images compared to humans. To explore this, we performed eye-tracking analyses of geneticist clinicians and non-clinicians. We compared results to DL-based saliency maps. We found that human visual attention when assessing images differs greatly from the parts of images weighted by the DL model. Further, individuals tend to have a specific pattern of image inspection, and clinicians demonstrate different visual attention patterns than non-clinicians.

show abstract

Counseling use case analysis of AI-based chatbot And Suggestion for developing a chatbot for School counseling

Cited by 4 publications

References 0 publications

Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in Japan

Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in Japan

Adoption of AI-Powered Chatbots with Large Language Models by Pathologists

Human and computer attention in assessing genetic conditions

Contact Info

Product

Resources

About