Text Mining: Extraction of Interesting Association Rule with Frequent Itemsets Mining for Korean Language from Unstructured Data

Abstract

Text mining is a specific method to extract knowledge from structured and unstructured data. This extracted knowledge from text mining process can be used for further usage and discovery. This paper presents the method for extraction information from unstructured text data and the importance of Association Rules Mining, specifically for of Korean language (text) and also, NLP (Natural Language Processing) tools are explained. Association Rules Mining (ARM) can also be used for mining association between itemsets from unstructured data with some modifications. Which can then, help for generating statistical thesaurus, to mine grammatical rules and to search large data efficiently. Although various association rules mining techniques have successfully used for market basket analysis but very few has applied on Korean text. A proposed Korean language mining method calculates and extracts meaningful patterns (association rules) between words and presents the hidden knowledge. First it cleans and integrates data, select relevant data then transform into transactional database. Then data mining techniques are used on data source to extract hidden patterns. These patterns are evaluated by specific rules until we get the valid and satisfactory result. We have tested on Korean news corpus and results have shown that it has worked well, and the results were adequate enough to research further.These processes are done until we get the required result. Technically, data mining is the process of finding correlations or patterns among dozens of fields in large relational databases. We have used same technique to extract meaningful association rules out of Korean text. First step is to pre-process data (getting rid of un-necessary data), then we transformed process data into transactional database. Grouping this transactional database with other hand-built databases we mined some interesting and useful patterns in given Korean text data collection (corpus).Rest of the paper is structured as follows: Section 2 covers related work; Section 3 describes the association rules mining for Korean language. Experiments and results are in Sections 4, and Sections 5 are conclusion and future work.

Data Analysis Method for Recognition of AI Education Based on Education Data

“…On the other hand, Opinion Mining is an analysis method to determine preferences such as positivity, negativity, and neutrality on structured and unstructured text collected from online media like social media or portal news [14]. It is applied appropriately to market size predictions, customer response, viral analysis, etc.…”

Analysis of Thesis Advising Pattern in Master Program on Faculty of Agriculture Ipb Using Data Mining Approach

“…Applying association rule mining in thesis document was done by Erman et al (2016) for finding term combination existed frequently and grouping summary thesis document. Khan et al (2015) applied association rule mining technique on Korean text document to extract a useful pattern between hidden word and information. Kulkarni et al (2016) reviewed the use of association rule mining in text mining to discover knowledge from a set of web documents.…”