Learning Quality Improved Word Embedding with Assessment of Hyperparameters

Word embedding approaches represent data sequences to handle their contextual meaning in the NLP tasks. Nowadays, there is an emerging need to understand the user behavior patterns over navigational clickstream data. However, representing the URL data sequences utilizing existing embedding approaches to cluster users' behavior with unsupervised machine learning tasks is a challenging task. This study introduces the Patter2Vec embedding approach using a representation vector to construct contextual, precise, and interpretable clusters over the hidden and popular navigational patterns. To test the usability of the proposed representation in clustering tasks, we conduct an experimental study, which indicates that Pattern2Vec outperforms existing embedding approaches.

Section: Literature Reviewmentioning

confidence: 93%

Section: Literature Reviewmentioning

confidence: 99%

Pattern2Vec: Representation of clickstream data sequences for learning user navigational behavior

Olmezogullari

Aktaş

2021

“…Various applications can use word embeddings such as natural language processing, social media, and the Internet of Things 19,20 . Our previous study, which we expanded in this article, explored the optimization of fewer hyperparameters using only accuracy metrics 21 …”

Section: Related Workmentioning

confidence: 99%

Improving word embedding quality with innovative automated approaches to hyperparameters

Yildiz

Tezgider

2021

Self Cite

Deep learning practices have a great impact in many areas. Big data and significant hardware developments are the main reasons behind deep learning success. Recent advances in deep learning have led to significant improvements in text analysis and classification. Progress in the quality of word representation is an important factor among these improvements. In this study, we aimed to develop word2vec word representation, also called embedding, by automatically optimizing hyperparameters. Minimum word count, vector size, window size, negative sample, and iteration number were used to improve word embedding. We introduce two approaches for setting hyperparameters that are faster than grid search and random search. Word embeddings were created using documents of approximately 300 million words. We measured the quality of word embedding using a deep learning classification model on documents of 10 different classes. It was observed that the optimization of the values of hyperparameters alone increased classification success by 9%. In addition, we demonstrate the benefits of our approaches by comparing the semantic and syntactic relations between word embedding using default and optimized hyperparameters.

“…28 Neural networks have been successfully used in other machine learning approaches to solve many problems such as forecasting, sentiment analysis, text classification, image processing. [29][30][31] Figure 1 illustrates the general structure of deep Q-learning. The neural networks are basically used to converge the optimal state-action value Q * .…”

Section: Deep Q-learningmentioning

confidence: 99%

“…Therefore, deep Q‐Learning, which is an off‐policy method for discrete action space, uses deep neural networks as an estimator instead of using a memory table 28 . Neural networks have been successfully used in other machine learning approaches to solve many problems such as forecasting, sentiment analysis, text classification, image processing 29–31 . Figure 1 illustrates the general structure of deep Q‐learning.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning using fully connected, attention, and transformer models in knapsack problem solving

Yildiz

2021

Self Cite

Knapsack is a combinatorial optimization problem that involves a variety of resource allocation challenges. It is defined as non-deterministic polynomial time (NP) hard and has a wide range of applications. Knapsack problem (KP) has been studied in applied mathematics and computer science for decades. Many algorithms that can be classified as exact or approximate solutions have been proposed. Under the category of exact solutions, algorithms such as branch-and-bound and dynamic programming and the approaches obtained by combining these algorithms can be classified. Due to the fact that exact solutions require a long processing time, many approximate methods have been introduced for knapsack solution. In this research, deep Q-learning using models containing fully connected layers, attention, and transformer as function estimators were used to provide the solution for KP. We observed that deep Q-networks, which continued their training by observing the reward signals provided by the knapsack environment we developed, optimized the total reward gained over time. The results showed that our approaches give near-optimum solutions and work about 40 times faster than an exact algorithm using dynamic programming.