LexScore: A Semantic Approach to Scoring Domain Specific Sentiment Lexicons

Kumar, Shantanu

doi:10.20944/preprints201906.0133.v1

Cited by 2 publications

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Evaluating the Applicability of Existing Lexicon-Based Sentiment Analysis Techniques on Family Medicine Resident Feedback Field Notes: Retrospective Cohort Study

Lu¹,

Meaney²,

Leung³

et al. 2023

JMIR Med Educ

View full text Add to dashboard Cite

Background Field notes, a form for resident-preceptor clinical encounter feedback, are widely adopted across Canadian medical residency training programs for documenting residents’ performance. This process generates a sizeable cumulative collection of feedback text, which is difficult for medical education faculty to navigate. As sentiment analysis is a subfield of text mining that can efficiently synthesize the polarity of a text collection, sentiment analysis may serve as an innovative solution. Objective This study aimed to examine the feasibility and utility of sentiment analysis using 3 popular sentiment lexicons on medical resident field notes. Methods We used a retrospective cohort design, curating text data from University of Toronto medical resident field notes gathered over 2 years (from July 2019 to June 2021). Lexicon-based sentiment analysis was applied using 3 standardized dictionaries, modified by removing ambiguous words as determined by a medical subject matter expert. Our modified lexicons assigned words from the text data a sentiment score, and we aggregated the word-level scores to a document-level polarity score. Agreement between dictionaries was assessed, and the document-level polarity was correlated with the overall preceptor rating of the clinical encounter under assessment. Results Across the 3 original dictionaries, approximately a third of labeled words in our field note corpus were deemed ambiguous and were removed to create modified dictionaries. Across the 3 modified dictionaries, the mean sentiment for the “Strengths” section of the field notes was mildly positive, while it was slightly less positive in the “Areas of Improvement” section. We observed reasonable agreement between dictionaries for sentiment scores in both field note sections. Overall, the proportion of positively labeled documents increased with the overall preceptor rating, and the proportion of negatively labeled documents decreased with the overall preceptor rating. Conclusions Applying sentiment analysis to systematically analyze field notes is feasible. However, the applicability of existing lexicons is limited in the medical setting, even after the removal of ambiguous words. Limited applicability warrants the need to generate new dictionaries specific to the medical education context. Additionally, aspect-based sentiment analysis may be applied to navigate the more nuanced structure of texts when identifying sentiments. Ultimately, this will allow for more robust inferences to discover opportunities for improving resident teaching curriculums.

show abstract

Evaluating the Applicability of Existing Lexicon-Based Sentiment Analysis Techniques on Family Medicine Resident Feedback Field Notes: Retrospective Cohort Study

Lu¹,

Meaney²,

Leung³

et al. 2023

JMIR Med Educ

View full text Add to dashboard Cite

show abstract

Evaluating the Applicability of Existing Lexicon-Based Sentiment Analysis Techniques on Family Medicine Resident Feedback Field Notes: Retrospective Cohort Study (Preprint)

Lu¹,

Meaney²,

Leung³

et al. 2022

Preprint

View full text Add to dashboard Cite

BACKGROUND Field notes are widely adopted across Canadian medical residency training programs for documenting resident performance feedback. This process generates a large cumulative collection of feedback text, which is difficult for medical education faculty to navigate. As sentiment analysis is a sub-field of text mining that can efficiently synthesize the polarity of a collection of text, sentiment analysis may serve as an innovative solution. OBJECTIVE To examine the utility and feasibility of sentiment analysis using three popular sentiment lexicons on medical resident field notes. METHODS We used a retrospective cohort design, curating text data from University of Toronto medical resident field note data gathered over a two-year period (July 2019 – June 2021). Lexicon-based sentiment analysis was applied using three standardized dictionaries, modified with the removal of ambiguous words as determined by a medical subject matter expert. Our modified lexicons assigned words from the text data a sentiment score, and aggregated word-level scores to a document-level polarity score. Agreement between dictionaries was assessed, and the document-level polarity was correlated with the overall preceptor rating of the clinical encounter under assessment. RESULTS Across the three original dictionaries, around a third of labeled words in our field note corpus were deemed ambiguous and removed to create modified dictionaries. Across all three modified dictionaries, the mean sentiment for the field note “Strengths” section was mildly positive, and slightly less positive in the “Areas of Improvement” section. We observed reasonable agreement between dictionaries for sentiment scores in both field note sections. Overall, the proportion of positively-labeled documents increased with the overall preceptor rating, and the proportion of negatively-labeled documents decreased with the overall preceptor rating. CONCLUSIONS Applying sentiment analysis to systematically analyze field notes is feasible. However, the applicability of existing lexicons is limited in the medical setting, even after the removal of ambiguous words. This warrants the need to generate new dictionaries specific to the context of medical education. Additionally, aspect-based sentiment analysis may be applied to navigate the more nuanced structure of texts when identifying sentiments. Ultimately, this will allow for stronger inferences shedding light on opportunities to advance resident teaching curriculums.

show abstract

LexScore: A Semantic Approach to Scoring Domain Specific Sentiment Lexicons

Cited by 2 publications

References 3 publications

Evaluating the Applicability of Existing Lexicon-Based Sentiment Analysis Techniques on Family Medicine Resident Feedback Field Notes: Retrospective Cohort Study

Evaluating the Applicability of Existing Lexicon-Based Sentiment Analysis Techniques on Family Medicine Resident Feedback Field Notes: Retrospective Cohort Study

Evaluating the Applicability of Existing Lexicon-Based Sentiment Analysis Techniques on Family Medicine Resident Feedback Field Notes: Retrospective Cohort Study (Preprint)

Contact Info

Product

Resources

About