On the effectiveness of testing sentiment analysis systems with metamorphic testing

Jiang, Mingyue; Chen, Tsong Yueh; Wang, Shuai

doi:10.1016/j.infsof.2022.106966

Cited by 12 publications

(3 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The MS Azure sentiment analysis service is used in [57] in order to calculate the sentiment of the mood that prevails in forum discussions regarding listed companies and include it in stock market data analysis. Jiang et al (2022) [58], assessed the effectiveness of MS Azure sentiment analysis service in conjunction with other sentiment analysis tools with metaphoric testing. In another study [59], the MS Azure sentiment machine learning was used to calculate the sentiment and satisfaction expressed by patients regarding online doctor services.…”

mentioning

confidence: 99%

A Fuzzy Synthetic Evaluation Approach to Assess Usefulness of Reviews by considering Bias Inherited in Sentiments and Articulacy

Kardaras,

Troussas,

Barbounaki

et al. 2024

Preprint

View full text Add to dashboard Cite

The reviews usefulness has been the aim of several research studies. However, results regarding the significance of usefulness determinants are often contradicting, thus decreasing the accuracy of reviews’ helpfulness estimation. Also, bias in user reviews attributed to differences e.g. in gender, nationality, etc., may result into misleading judgments thus diminishing reviews’ usefulness. Research is needed for sentiment analysis algorithms that incorporate bias embedded in reviews, thus improving their usefulness, readability, credibility, etc. This study utilizes fuzzy relations and fuzzy synthetic evaluation (FSE) in order to calculate reviews’ usefulness by incorporating users’ biases as expressed in terms of reviews’ articulacy and sentiment polarity. It selected and analysed 95.678 hotel user reviews from Tripadvisor, for five nationalities. The findings indicate that there are differences among nationalities. The British are most consistent in their judgments expressed in titles and review documents. The British and the Greek review titles suffice to convey any negative sentiments. The Dutch use fewer words in their reviews than the other nationalities. This study suggests that fuzzy logic captures subjectivity which is often found in reviews, and it can be used to quantify users’ behavioral differences, calculate reviews usefulness, and provide the means for developing more accurate voting systems.

show abstract

mentioning

confidence: 99%

A Fuzzy Synthetic Evaluation Approach to Assess Usefulness of Reviews by considering Bias Inherited in Sentiments and Articulacy

Kardaras,

Troussas,

Barbounaki

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Subsequently, it included the results in stock market data analysis. Jiang et al (2022) [58] assessed the effectiveness of the MS Azure sentiment analysis service in conjunction with other sentiment analysis tools via metaphoric testing. In another study [59], MS Azure sentiment machine learning was used to calculate the sentiment and satisfaction expressed by patients regarding online doctor services.…”

mentioning

confidence: 99%

“…The same MS Azure sentiment analysis service was used in[57] to calculate the sentiment of the mood that prevails in forum discussions regarding listed companies. Subsequently, it included the results in stock market data analysis Jiang et al (2022) [58]. assessed the effectiveness of the MS Azure sentiment analysis service in conjunction with other sentiment analysis tools via metaphoric testing.…”

mentioning

confidence: 99%

A Fuzzy Synthetic Evaluation Approach to Assess Usefulness of Tourism Reviews by Considering Bias Identified in Sentiments and Articulacy

Kardaras,

Troussas,

Barbounaki

et al. 2024

Information

View full text Add to dashboard Cite

Assessing the usefulness of reviews has been the aim of several research studies. However, results regarding the significance of usefulness determinants are often contradictory, thus decreasing the accuracy of reviews’ helpfulness estimation. Also, bias in user reviews attributed to differences, e.g., in gender, nationality, etc., may result in misleading judgments, thus diminishing reviews’ usefulness. Research is needed for sentiment analysis algorithms that incorporate bias embedded in reviews, thus improving their usefulness, readability, credibility, etc. This study utilizes fuzzy relations and fuzzy synthetic evaluation (FSE) in order to calculate reviews’ usefulness by incorporating users’ biases as expressed in terms of reviews’ articulacy and sentiment polarity. It selected and analyzed 95,678 hotel user reviews from Tripadvisor, written by users from five specific nationalities. The findings indicate that there are differences among nationalities in terms of the articulacy and sentiment of their reviews. The British are most consistent in their judgments expressed in titles and the main body of reviews. For the British and the Greeks, review titles suffice to convey any negative sentiments. The Dutch use fewer words in their reviews than the other nationalities. This study suggests that fuzzy logic captures subjectivity which is often found in reviews, and it can be used to quantify users’ behavioral differences, calculate reviews’ usefulness, and provide the means for developing more accurate voting systems.

show abstract

Metamorphic testing of chess engines

Benito-Parejo

Ibias

Núñez

2023

Information and Software Technology

View full text Add to dashboard Cite

On the effectiveness of testing sentiment analysis systems with metamorphic testing

Cited by 12 publications

References 43 publications

A Fuzzy Synthetic Evaluation Approach to Assess Usefulness of Reviews by considering Bias Inherited in Sentiments and Articulacy

A Fuzzy Synthetic Evaluation Approach to Assess Usefulness of Reviews by considering Bias Inherited in Sentiments and Articulacy

A Fuzzy Synthetic Evaluation Approach to Assess Usefulness of Tourism Reviews by Considering Bias Identified in Sentiments and Articulacy

Metamorphic testing of chess engines

Contact Info

Product

Resources

About