LSA &amp; LDA topic modeling classification: comparison study on e-books

Mohammed, Shaymaa H.; Al-augby, Salam

doi:10.11591/ijeecs.v19.i1.pp353-362

Cited by 44 publications

(22 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Latent Dirichlet Allocation (LDA) is a generative statistical model used to extract the latent topic structure of text documents. LDA is a machine-learning technique used in different areas such as retrieval field, document classification and topic modelling [ 31 ].…”

Section: Methodsmentioning

confidence: 99%

Finding answers to COVID-19-specific questions: An information retrieval system based on latent keywords and adapted TF-IDF

Chamorro-Padial

Rodrigo-Ginés

Rodrı́guez-Sánchez

2022

Journal of Information Science

View full text Add to dashboard Cite

The scientific community has reacted to the COVID-19 outbreak by producing a high number of literary works that are helping us to understand a variety of topics related to the pandemic from different perspectives. Dealing with this large amount of information can be challenging, especially when researchers need to find answers to complex questions about specific topics. We present an Information Retrieval System that uses latent information to select relevant works related to specific concepts. By applying Latent Dirichlet Allocation (LDA) models to documents, we can identify key concepts related to a specific query and a corpus. Our method is iterative in that, from an initial input query defined by the user, the original query is expanded for each subsequent iteration. In addition, our method is able to work with a limited amount of information per article. We have tested the performance of our proposal using human validation and two evaluation strategies, achieving good results in both of them. Concerning the first strategy, we performed two surveys to determine the performance of our model. For all the categories that were studied, precision was always greater than 0.6, while accuracy was always greater than 0.8. The second strategy also showed good results, achieving a precision of 1.0 for one category and scoring over 0.7 points overall.

show abstract

Section: Methodsmentioning

confidence: 99%

Finding answers to COVID-19-specific questions: An information retrieval system based on latent keywords and adapted TF-IDF

Chamorro-Padial

Rodrigo-Ginés

Rodrı́guez-Sánchez

2022

Journal of Information Science

View full text Add to dashboard Cite

show abstract

“…Dan setiap topik laten merupakan character atau kata yang didistribusikan dan mewakili keseluruhan kata dalam dokumen. Topik laten dapat dihasilkan dari kumpulan kata pada dokumen dengan nilai proporsi yang berbeda [12]. Data yang ada terlebih dahulu dibuat menjadi corpus atau Dictionary dengan memanfaatkan modul library Gensim pada Python, yaitu gensim.corpora.…”

Section: Membangun Model Ldaunclassified

Analisis Topik Modelling Terhadap Penggunaan Sosial Media Twitter oleh Pejabat Negara

Patmawati

Yusuf

2021

bits

View full text Add to dashboard Cite

Social media is experiencing rapid development until now. Social media makes it easy for humans to be able to connect with one another. One of the social media that is widely used is Twitter. The ease of use makes social media widely used among the public, including state officials. State officials use Twitter to convey policies, opinions and interact with the public. By conducting a topic analysis of tweets shared by state officials, we can find out the relevant topics discussed by state officials. We can find out the focus of attention of state officials through topic modeling. Latent Dirichlet Allocation (LDA) is a topic modeling method that finds certain patterns in documents and produces several different topics. Tweets from the @jokowi account are collected using a scraping technique. The results of the tweet collection are then preprocessed for further analysis using the Latent Dirichlet Allocation (LDA) method. The results of the analysis model are evaluated using perplexity calculations and coherence scores. The evaluation of the model resulted in a perplexity value of -8.069 and a coherence score of 0.375 for a total of 7. This shows that the model used is good for analyzing and finding topics in tweets of state officials.

show abstract

“…Pada penelitian ini digunakan Latent Semantic Indexing (LSI), Latent Dirichlet Allocation (LDA), dan Hierarchical Dirichlet Process (HDP) sebagai metode pemodelan topik. LSI atau disebut juga Latent Semantic Analysis (LSA) merupakan metode ekstraksi dan representasi kata dari teks dengan menggunakan perhitungan statis pada dokumen atau data berukuran besar (Mohammed & Al-augby, 2020). LSA merupakan metode yang cepat dan paling mudah digunakan diantara metode pemodelan topik (Qomariyah et al, 2019).…”

Section: Pendahuluanunclassified

Eliminasi Non-Topic Menggunakan Pemodelan Topik untuk Peringkasan Otomatis Data Tweet dengan Konteks Covid-19

Damayanti

Purwitasari

Suciati

2021

JTIIK

View full text Add to dashboard Cite

Akun twitter, seperti Suara Surabaya, dapat membantu menyebarkan informasi tentang COVID-19 meskipun ada bahasan lainnya seperti kecelakaan, kemacetan atau topik lain. Peringkasan teks dapat diimplementasikan pada kasus pembacaan data twitter karena banyaknya jumlah tweet yang tersedia, sehingga akan mempermudah dalam memperoleh informasi penting terkini terkait COVID-19. Jumlah variasi bahasan pada teks tweet mengakibatkan hasil ringkasan yang kurang baik. Oleh karena itu dibutuhkan adanya eliminasi tweet yang tidak berkaitan dengan konteks sebelum dilakukan peringkasan. Kontribusi penelitian ini adalah adanya metode pemodelan topik sebagai bagian tahapan dalam serangkaian proses eliminasi data. Metode pemodelan topik sebagai salah satu teknik eliminasi data dapat digunakan dalam berbagai kasus namun pada penelitian ini difokuskan pada COVID-19. Tujuannya adalah untuk mempermudah masyarakat memperoleh informasi terkini secara ringkas. Tahapan yang dilakukan adalah pra-pemrosesan, eliminasi data menggunakan pemodelan topik dan peringkasan otomatis. Penelitian ini menggunakan kombinasi beberapa metode word embedding, pemodelan topik dan peringkasan otomatis sebagai pembanding. Ringkasan diuji menggunakan metode ROUGE dari setiap kombinasi untuk ditemukan kombinasi terbaik dari penelitian ini. Hasil pengujian menunjukkan kombinasi metode Word2Vec, LSI dan TextRank memiliki nilai ROUGE terbaik yaitu 0.67. Sedangkan kombinasi metode TFIDF, LDA dan Okapi BM25 memiliki nilai ROUGE terendah yaitu 0.35. AbstractTwitter accounts, such as Suara Surabaya, can help spread information about COVID-19 even though there are other topics such as accidents, traffic jams or other topics. Text summarization can be implemented in the case of reading Twitter data because of the large number of tweets available, making it easier to obtain the latest important information related to COVID-19. The number of discussion variations in the tweet text results in poor summary results. Therefore, it is necessary to eliminate tweets that are not related to the context before summarization is carried out. The contribution to this research is the topic modeling method as part of a series of data elimination processes. The topic modeling method as a data elimination technique can be used in various cases, but this research focuses on COVID-19. The aim is to make it easier for the public to obtain current information in a concise manner. The steps taken in this study were pre-processing, data elimination using topic modeling and automatic summarization. This study uses a combination of several word embedding methods, topic modeling and automatic summarization as a comparison. The summary is tested using the ROUGE method of each combination to find the best combination of this study. The test results show that the combination of Word2Vec, LSI and TextRank methods has the best ROUGE value, 0.67. While the combination of TFIDF, LDA and Okapi BM25 methods has the lowest ROUGE value, 0.35.

show abstract

LSA & LDA topic modeling classification: comparison study on e-books

Cited by 44 publications

References 15 publications

Finding answers to COVID-19-specific questions: An information retrieval system based on latent keywords and adapted TF-IDF

Finding answers to COVID-19-specific questions: An information retrieval system based on latent keywords and adapted TF-IDF

Analisis Topik Modelling Terhadap Penggunaan Sosial Media Twitter oleh Pejabat Negara

Eliminasi Non-Topic Menggunakan Pemodelan Topik untuk Peringkasan Otomatis Data Tweet dengan Konteks Covid-19

Contact Info

Product

Resources

About