Sentence boundary detection of various forms of Tunisian Arabic

Mekki, Asma; Zribi, Inès; Ellouze, Mariem; Belguith, Lamia Hadrich

doi:10.1007/s10579-021-09538-4

Cited by 4 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, we used [9] to add a Part Of Speech (POS) tag for each word of the data set. We applied the segmentor provided by [10] since the input for POS tagging needed to be segmented. We also added the month and year of the comment.…”

Section: Data Annotationmentioning

confidence: 99%

A Tunisian benchmark social media data set for COVID-19 sentiment analysis and sarcasm detection

Mekki

Zribi

Ellouze

et al. 2022

Preprint

View full text Add to dashboard Cite

The aim of this research is to better understand public perceptions of COVID-19 pandemic patterns and to identify key themes of concern expressed by Tunisian dialect social media users throughout the epidemic. We collected around 23K comments written in Tunisian dialect in both Arabic and Latin letters. These comments were manually annotated by native language experts for sentiment analysis (optimist, pessimist and neutral) and sarcasm detection (sarcastic and non-sarcastic). In addition to health, our data set includes comments relating to additional COVID-19-influenced thematic areas, such as entertainment, social, sports, religion and politics. This paper deals with an extensive analysis of the sentiments and sarcasm expressed in Tunisian social media comments about the novel COVID-19 since its release at the beginning of 2020. On the data set, we also report benchmarking results for sentiment analysis and sarcasm detection using machine learning and deep learning techniques. The best models achieved an accuracy of above 70% on both sentiment analysis and sarcasm detection.

show abstract

Section: Data Annotationmentioning

confidence: 99%

A Tunisian benchmark social media data set for COVID-19 sentiment analysis and sarcasm detection

Mekki

Zribi

Ellouze

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Depending on the form of the data, SBD tasks can be divided into two categories. Text containing punctuation marks is processed by sentence boundary disambiguation, while sentence boundary detecction is performed for text without punctuation marks [12] . Corpus in this study contains punctuation masks, so SBD in this study is defined as sentence boundary disambiguation.…”

Section: Introductionmentioning

confidence: 99%

A Tibetan Sentence Boundary Disambiguation Model Considering the Components on Information on Both Sides of Shad

Gao

et al. 2023

Tsinghua Sci. Technol.

View full text Add to dashboard Cite

Sentence Boundary Disambiguation (SBD) is a preprocessing step for natural language processing.Segmenting text into sentences is essential for Deep Learning (DL) and pretraining language models. Tibetan punctuation marks may involve ambiguity about the sentences' beginnings and endings. Hence, the ambiguous punctuation marks must be distinguished, and the sentence structure must be correctly encoded in language models. This study proposed a component-level Tibetan SBD approach based on the DL model. The models can reduce the error amplification caused by word segmentation and part-of-speech tagging. Although most SBD methods have only considered text on the left side of punctuation marks, this study considers the text on both sides. In this study, 465 669 Tibetan sentences are adopted, and a Bidirectional Long Short-Term Memory (Bi-LSTM) model is used to perform SBD. The experimental results show that the F1-score of the Bi-LSTM model reached 96%, the most efficient among the six models. Experiments are performed on low-resource languages such as Turkish and Romanian, and high-resource languages such as English and German, to verify the models' generalization.

show abstract