Automatic Bengali Document Categorization Based on Deep Convolution Nets

Hossain, Md. Rajib; Hoque, Mohammed Moshiul

doi:10.1007/978-981-13-5953-8_43

Cited by 16 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bangyal et al [112] [122] employed a deep convolution neural network (DCNN) for both feature extraction and relation classification. The DCNN model is comprised of three convolution layers of unique filters, i.e., 16, 32, and 64, with activation and max pooling layers coming after each convolution layer, followed by two fully connected layers with 512 neurons each and a softmax layer which does the classification using the sigmoid activation [121], the DCNN architecture is shown in Figure 18. Each word in the text is passed through the look-up table (W v.d ) to form the input document representation vector n * d, where n is the document's word count and d is its feature dimension.…”

Section: Relation Discovery In Ontology Learning Involves Attribute O...mentioning

confidence: 99%

“…Each word in the text is passed through the look-up table (W v.d ) to form the input document representation vector n * d, where n is the document's word count and d is its feature dimension. Once the input tensor extracts the local feature, the convolution layer learns the filter weights [121]. Furthermore, ReLU is used in deep convolution neural networks (DCNNs) to introduce nonlinearity and rectification layers for computation of gradient, mitigation of vanishing gradient problem, and reduction of computational time complexity [123].…”

Section: Relation Discovery In Ontology Learning Involves Attribute O...mentioning

confidence: 99%

See 1 more Smart Citation

A Review of State of the Art Deep Learning Models for Ontology Construction

Zengeya,

Vincent Fonou-Dombeu

2024

IEEE Access

View full text Add to dashboard Cite

ABSRTACT Researchers are working towards automation of ontology construction to manage the evergrowing data on the web. Currently, there is a shift from the use of machine learning techniques towards exploration of deep learning models for ontology construction. Deep learning model are capable of extracting terms, entities, relations, and classifications, and perform axiom learning from the underutilized richness of web-based knowledge. There has been remarkable progress in automatic ontology creation using deep learning models since they can perform word embedding, long-term dependency acquisition, concept extraction from large corpora, and inference of abstracted relationships based on broad corpora. Despite their emerging importance, deep learning models remain underutilized in ontology construction, and there is no comprehensive review of their application in ontology learning. This paper presents a comprehensive review of existing deep-learning models for the construction of ontologies, the strength and the weaknesses presented by the deep learning models for ontology learning as well as promising directions to achieve a more robust deep learning models. The Deep Learning models reviewed include Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Long-Short Term Memory (LSTMs), and Gated Recurrent Unit (GRU) as well as their ensembles. While these traditional deep learning models have achieved great success, one of their limitations is that they struggle to understand the meaning and order of data in sequences. CNNs and RNN-based models such as LSTMs and GRUs can be computationally expensive due to their large number of parameters or complex gating mechanisms. Furthermore, RNN models suffer from vanishing gradients, making it difficult to learn long-term relationships in sequences. Additionally, RNN-based models process information sequentially, limiting their ability to take advantage of powerful parallel computing hardware, slowing down training and inference, especially for long sequences. Consequently, there has been a shift towards Generative Pre-Trained (GPT) models and Bidirectional Encoder Representations from Transformers (BERT) models. This paper also reviewed the GPT-3, GPT-4, and the BERT models for extracting terms, entities, relations, and classifications. While GPT models excel in contextual understanding and flexibility, they fall short when handling domain-specific terminology and disambiguating complex relationships. Fine-tuning and domain-specific training data could minimize these shortcomings, and further enhance the performance of GPT in term and relation extraction tasks. On the other hand, the BERT models excel in comprehending context-heavy texts, but struggles with higher-level abstraction and inference tasks due to a lack of explicit semantic knowledge, thus necessitating inference for unspecified relationships. The paper recommends further research on deep learning models for ontology alignment and merging. Also, the ensembling of deep learning models and the u...

show abstract

Section: Relation Discovery In Ontology Learning Involves Attribute O...mentioning

confidence: 99%

Section: Relation Discovery In Ontology Learning Involves Attribute O...mentioning

confidence: 99%

A Review of State of the Art Deep Learning Models for Ontology Construction

Zengeya,

Vincent Fonou-Dombeu

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The intrinsic and extrinsic evaluations are used for evaluating the embedding model. The intrinsic evaluators evaluate the semantic, syntactic and relatedness quality whereas the extrinsic evaluators evaluate the downstream tasks e.g., classification [14], machine translation [61], word-sense disambiguation [62], and question answering [63]. Spearman (ρ) and Pearson (r) correlations are used for intrinsic evaluation such as semantic word similarity (S sρ /S sr ) and syntactic word similarity (S yρ /S yr ).…”

Section: A Evaluation Measuresmentioning

confidence: 99%

Authorship Classification in a Resource Constraint Language Using Convolutional Neural Networks

et al. 2021

Self Cite

View full text Add to dashboard Cite

Authorship classification is a technique of automatically determining the appropriate author of an unknown linguistic text. Although research on authorship classification has significantly progressed in high-resource languages, it is at a primitive stage in the realm of resource-constraint languages like Bengali. This paper presents an authorship classification system made of Convolution Neural Networks (CNN) comprising four modules: embedding model generation, feature representation, classifier training and classifier testing. For this purpose, this work develops a new embedding corpus (named WEC) and a Bengali authorship classification corpus (called BACC-18), which are more robust in terms of authors' classes and unique words. Using three text embedding techniques (Word2Vec, GloVe and FastText) and combinations of different hyperparameters, 90 embedding models are created in this study. All the embedding models are assessed by intrinsic evaluators and selected the best 9 performing models out of the 90 models for the authorship classification. In total 36 classification models, including four classification models (CNN, LSTM, SVM, SGD) and three embedding techniques with 100, 200 and 250 embedding dimensions, are trained with optimized hyperparameters and tested on three benchmark datasets BAAD16 and LD). Among the models, the optimized CNN with GloVe model achieved the highest classification accuracies of 93.45%, 95.02%, and 98.67% for the datasets BACC-18, BAAD16, and LD, respectively.INDEX TERMS Natural language processing, Authorship classification, resource constraint language, semantic feature extraction, deep learning.

show abstract

Semantic Meaning Based Bengali Web Text Categorization Using Deep Convolutional and Recurrent Neural Networks (DCRNNs)

Hossain

Hoque

2021

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Automatic Bengali Document Categorization Based on Deep Convolution Nets

Cited by 16 publications

References 10 publications

A Review of State of the Art Deep Learning Models for Ontology Construction

A Review of State of the Art Deep Learning Models for Ontology Construction

Authorship Classification in a Resource Constraint Language Using Convolutional Neural Networks

Semantic Meaning Based Bengali Web Text Categorization Using Deep Convolutional and Recurrent Neural Networks (DCRNNs)

Contact Info

Product

Resources

About