Weight Dropout for Preventing Neural Networks from Overfitting

Sanjar, Karshiev; Rehman, Abdul; Paul, Anand; Kim, Jeonghong

doi:10.1109/icot51877.2020.9468799

Cited by 19 publications

(8 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Compared with the complexity of the model, the training dataset is small, which makes the model prone to overfitting. At present, researchers usually use methods such as expanding dataset, removing features, regularization, and terminating training in advance to prevent model overfitting ( Sanjar et al, 2020 ). Data enhancement is a way to increase training data, which can be realized by flipping, translation, rotation, scaling, and generation methods.…”

Section: Discussionmentioning

confidence: 99%

Hierarchical Spatiotemporal Electroencephalogram Feature Learning and Emotion Recognition With Attention-Based Antagonism Neural Network

et al. 2021

View full text Add to dashboard Cite

Inspired by the neuroscience research results that the human brain can produce dynamic responses to different emotions, a new electroencephalogram (EEG)-based human emotion classification model was proposed, named R2G-ST-BiLSTM, which uses a hierarchical neural network model to learn more discriminative spatiotemporal EEG features from local to global brain regions. First, the bidirectional long- and short-term memory (BiLSTM) network is used to obtain the internal spatial relationship of EEG signals on different channels within and between regions of the brain. Considering the different effects of various cerebral regions on emotions, the regional attention mechanism is introduced in the R2G-ST-BiLSTM model to determine the weight of different brain regions, which could enhance or weaken the contribution of each brain area to emotion recognition. Then a hierarchical BiLSTM network is again used to learn the spatiotemporal EEG features from regional to global brain areas, which are then input into an emotion classifier. Especially, we introduce a domain discriminator to work together with the classifier to reduce the domain offset between the training and testing data. Finally, we make experiments on the EEG data of the DEAP and SEED datasets to test and compare the performance of the models. It is proven that our method achieves higher accuracy than those of the state-of-the-art methods. Our method provides a good way to develop affective brain–computer interface applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Hierarchical Spatiotemporal Electroencephalogram Feature Learning and Emotion Recognition With Attention-Based Antagonism Neural Network

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Overfitting pada model dapat dicegah melalui beberapa cara, diantaranya seperti Dropout [4], Regularization [5], dan Batch Normalization [6]. Selain beberapa solusi tersebut, overfitting pada model dapat dicegah dengan menggunakan dataset dengan jumlah yang besar dan bervariasi.…”

Section: Pendahuluanunclassified

Augmentasi Data pada Prasasti Logam untuk Deteksi Aksara Kawi

Rachmat Santoso

2024

View full text Add to dashboard Cite

Deep learning telah diterapkan di berbagai bidang seperti pembacaan aksara pada prasasti. Beberapa penelitian telah menerapkan deep learning dan menciptakan model dengan akurasi yang tinggi untuk melakukan deteksi dan pengenalan aksara pada prasasti. Seperti diketahui deep learning memerlukan data dalam jumlah yang besar dan bervariasi untuk menghasilkan model dengan performa yang tinggi. Namun sayangnya, tidak semua data memiliki jumlah yang cukup besar untuk mendukung proses pelatihan dengan deep learning. Data Prasasti merupakan salah satu contoh data yang memiliki jumlah terbatas karena susah untuk ditemukan. Untuk mendukung ketersediaan data dalam jumlah besar serta bervariasi, penelitian ini menerapkan augmentasi pada data aksara di prasasti untuk memenuhi kebutuhan dataset pelatihan deep learning. Teknik augmentasi data yang digunakan telah disesuaikan dengan data berjenis aksara, seperti rotasi, translasi, zoom, blur serta kombinasi antara teknik tersebut untuk menciptakan variasi baru. Penelitian ini menggunakan 3 (tiga) sampel data gambar yang diambil dari salah satu lempeng Prasasti Waruṅgahan. Menerapkan 19 teknik augmentasi data, penelitian ini menghasilkan data baru dengan jumlah yang besar serta bervariasi. Sebelum augmentasi, data gambar hanya sebanyak 3 (tiga) data dengan total 285 label. Setelah augmentasi dilakukan, jumlah data meningkat menjadi 60 data gambar lempeng dengan total 4975 label. Hasil augmentasi data ini digunakan sebagai dataset pelatihan untuk YOLO object detection. Hasil pelatihan yang dilakukan pada penelitian lain menunjukkan bahwa penambahan jumlah serta variasi data meningkatkan akurasi pendeteksian aksara khususnya pada Prasasti Waruṅgahan.

show abstract

“…The adam optimizer [26] is the one that is most frequently used. Furthermore, to avoid the ANN model overfitting, we applied the dropout strategy [27], which involves briefly removing neurons from the hidden layers during model training. Figure 2 shows the structure of the ANN model with the best hyper-parameters values for the dysgraphia dataset determined by the keras-tuner.…”

Section: Artificial Neural Network Modelmentioning

confidence: 99%

“…Our model is made up of an input layer with 15 neurons representing dataset dimensions and three hidden layers, each with 22 neurons, 26 neurons, and 14 neurons. We used two dropout layers (with p=0.5) after each of the two first fully connected hidden layers, as proposed in the original paper [27], and an output layer with one neuron for binary dysgraphia classifying.…”

Section: Artificial Neural Network Modelmentioning

confidence: 99%

Digital handwriting characteristics for dysgraphia detection using artificial neural network

Ikermane

Mouatasim

2023

Bulletin EEI

View full text Add to dashboard Cite

Despite all of the technical advancements in writing and text editing with keyboards on numerous devices, writing with a pen remains a fundamental ability in modern human existence. Handwriting disabilities are referred to as dysgraphia. Nonetheless, how well they are taught to write in school, 10-30% of children never attain a respectable level of handwriting. Early identification is critical because it can help children avoid difficulties in their behavioral and academic development. On blank papers attached to digital tablets, 280 individuals were asked to complete the concise evaluation scale for children’s handwriting (BHK), with 218 having typical handwriting and 62 having dysgraphia. In addition to their age and BHK quality and speed scores, 12 variables identifying digital handwriting across several domains (static, kinematic, pressure, and tilt) were collected. In this paper, we provided a rapid and automated dysgraphia classification approach using an artificial neural network (ANN) model. Using digital handwriting traits as an input to the ANN approach, the prediction findings were encouraging and very accurate, reaching 96% accuracy, and they could lead to the development of a new self-administered dysgraphia screening tool.

show abstract

Weight Dropout for Preventing Neural Networks from Overfitting

Cited by 19 publications

References 2 publications

Hierarchical Spatiotemporal Electroencephalogram Feature Learning and Emotion Recognition With Attention-Based Antagonism Neural Network

Hierarchical Spatiotemporal Electroencephalogram Feature Learning and Emotion Recognition With Attention-Based Antagonism Neural Network

Augmentasi Data pada Prasasti Logam untuk Deteksi Aksara Kawi

Digital handwriting characteristics for dysgraphia detection using artificial neural network

Contact Info

Product

Resources

About