A comparative study of TF*IDF, LSI and multi-words for text classification

Zhang, Wen; Yoshida, Taketoshi; Tang, Xijin

doi:10.1016/j.eswa.2010.08.066

Cited by 552 publications

(243 citation statements)

References 22 publications

Supporting

Mentioning

233

Contrasting

Unclassified

Order By: Relevance

“…fields. Our analyses as well as findings are consistent with the theoretical accounts that words and key terms, both single and multiword key terms, collectively represent the content of the text and that they can be analyzed statistically while maintaining semantic quality (Zhang, Yoshida, & Tang, 2011). The fact that we used TF-IDF as a key term selection method (hence the emphasis placed on the importance of a term in a document) and the advanced option is checked for multiword (hence the contextual information of individual words is captured) (see Figure 2), and later the multiword list is used to transform the entire corpus increases the reliability of co-occurrence analysis to reveal the structure of the scientific literature and, in turn, the fields under investigation.…”

Section: Discussionsupporting

confidence: 75%

A bibliometric mapping of the structure of STEM education using co‐word analysis

Assefa

Rorissa

2013

J Am Soc Inf Sci Tec

View full text Add to dashboard Cite

STEM, a set of fields that includes science, technology, engineering, and mathematics; allied disciplines ranging from environmental, agricultural, and earth sciences to life science and computer science; and education and training in these fields, is clearly at the top of the list of priority funding areas for governments, including the United States government. The U.S. has 11 federal agencies dedicated to supporting programs and providing funding for research and curriculum development. The domain of STEM education has significant implications in preparing the desired workforce with the requisite knowledge, developing appropriate curricula, providing teachers the necessary professional development, focusing research dollars on areas that have maximum impact, and developing national educational policy and standards. A complex undertaking such as STEM education, which attracts interest and valuable resources from a number of stakeholders needs to be well understood. In light of this, we attempt to describe the underlying structure of STEM education, its core areas, and their relationships through co-word analyses of the titles, keywords, and abstracts of the relevant literature using visualization and bibliometric mapping tools. Implications are drawn with respect to the nature of STEM education as well as curriculum and policy development.

show abstract

Section: Discussionsupporting

confidence: 75%

A bibliometric mapping of the structure of STEM education using co‐word analysis

Assefa

Rorissa

2013

J Am Soc Inf Sci Tec

View full text Add to dashboard Cite

show abstract

“…This model can be extended by using n-grams [2,9,44]. We use the bag-of-words model with a combination of unigrams and bigrams.…”

Section: Puls Overviewmentioning

confidence: 99%

Improving Supervised Classification Using Information Extraction

Pierce

Pivovarova

et al. 2015

Natural Language Processing and Information Systems

View full text Add to dashboard Cite

Abstract. We explore supervised learning for multi-class, multi-label text classification, focusing on real-world settings, where the distribution of labels changes dynamically over time. We use the PULS Information Extraction system to collect information about the distribution of class labels over named entities found in text. We then combine a knowledge-based rote classifier with statistical classifiers to obtain better performance than either classification method alone. The resulting classifier yields a significant improvement in macro-averaged F-measure compared to the state of the art, while maintaining comparable micro-average.

show abstract

“…The authors used back-propagation neural networks for this purpose. Zhang et al (2011) presented some experimental evaluations of indexing methods on text classification and analyzed that presently we do not have a standard measure to assess the semantic and statistical qualities of text. Attia et al (2014) proposed a linguistic-based multi-view fuzzy ontology information retrieval model that allows the users to define all their linguistic terms according to their subjective view which helps in retrieving documents according to their linguistic term definitions not to our definitions.…”

Section: Background and Literature Reviewmentioning

confidence: 99%

A novel fuzzy document-based information retrieval scheme (FDIRS)

Roy

2016

Appl Inform

View full text Add to dashboard Cite

BackgroundPrediction or forecasting is both an art as well as science. The process and outcome of forecasting have long been a matter of research and still are in its childhood state. We can devise numerous ways of modeling a phenomenon and predict its outcome, but there are no universal methods using which we can model every phenomena. Modeling of linear systems is comparatively simpler than dynamical systems. Stock markets are completely chaotic and dynamic systems which are both time and sentiment driven. The time series generated through stock market data can only represent a financial time series of prices but cannot represent the overall sentiment of the market players who trade and invest in the stock markets. Hence modeling of stock market data is one of the toughest as it should incorporate not only data but market sentiment also. The stock market data are a series of prices that are observed in a series of certain time intervals (minutes, hours, days, or weeks etc.). Data mining is a very effective tool using which the Abstract Information retrieval systems are generally used to find documents that are most appropriate according to some query that comes dynamically from the users. In this paper, a novel fuzzy document-based information retrieval scheme (FDIRS) is proposed for the purpose of Stock Market Index forecasting. The novelty of the proposed approach is the use of a modified tf-idf scoring scheme to predict the future trend of the stock market index. The contribution of this paper has two dimensions: (1) In the proposed system, the simple daily time series data are converted to an enriched fuzzy linguistic time series with a unique approach of incorporating information about the manner in which the OHLC (open, high, low, and close) price formation took place at every instance of the time series, and (2) A unique approach is followed while modeling the information retrieval (IR) system which converts a simple IR system into a forecasting system. The modified IR system provides us with a trend forecast and after which a crisp value is generated that becomes the forecast value that can be achieved in next few trading sessions. From the performance comparison of FDIRS with standard benchmark models, it can be affirmed that the proposed model has a potential of becoming a good forecasting model. Transaction data of CNX NIFTY-50 index of National Stock Exchange of India are used to experiment and validate the proposed model. Roy Appl Inform (2016) past behavior of the price movement can be modeled to predict the future. Fuzzy logic is a very effective tool using which the market sentiment can be captured and modeled. By adopting a hybrid approach of combining time series, data mining, and fuzzy logic, an effective system can be built to model the stock market price data that can not only give information about price but also the market sentiment or the mood of the market participants. Keywords RESEARCHThe stock market gives facilities to gain both from rising prices as well as from falling prices....

show abstract

A comparative study of TF*IDF, LSI and multi-words for text classification

Cited by 552 publications

References 22 publications

A bibliometric mapping of the structure of STEM education using co‐word analysis

A bibliometric mapping of the structure of STEM education using co‐word analysis

Improving Supervised Classification Using Information Extraction

A novel fuzzy document-based information retrieval scheme (FDIRS)

Contact Info

Product

Resources

About