Ideal Modified Adachi Chaotic Neural Networks and active shape model for infant facial cry detection on still image

Kristian, Yosi; Hariadi, Mochamad; Purnomo, Mauridhi Hery

doi:10.1109/ijcnn.2014.6889520

Cited by 4 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The method of using autoencoder for unsupervised learning has been used previously by another study. In our previous study 11,12 a cry detection and pain classification system was created. This study used 46 videos taken by E. Hanindito et al 18 and added ten more videos with multiple duration.…”

Section: Discussionmentioning

confidence: 99%

“…In 2018, while deep learning had already been widely used as a feature learning method, Y. Kristian, et al 39 removed all the hand-crafted feature extraction, and switched to deep learning by using autoencoders. This research still used the same ASM from their previous research 12 to crop the face region and discarded unimportant data. This paper transforms the ASM model into Kazemi et al face landmark algorithm.…”

Section: Discussionmentioning

confidence: 99%

“…We also use a pre-train representation of unsupervised learning, such as autoencoders to overcome the limited labeled data in this study. This research used the ASM model from previous research 12 to crop the face region and discarded unimportant data.…”

Section: Multimodal Pain and Cry Detectionmentioning

confidence: 99%

See 2 more Smart Citations

Ensemble of multimodal deep learning autoencoder for infant cry and pain detection

Kristian¹,

Simogiarto²,

Sampurna

et al. 2023

F1000Res

View full text Add to dashboard Cite

Background: Babies cannot communicate their pain properly. Several pain scores are developed, but they are subjective and have high variability inter-observer agreement. The aim of this study was to construct models that use both facial expression and infant voice in classifying pain levels and cry detection. Methods: The study included a total of 23 infants below 12-months who were treated at Dr Soetomo General Hospital. The the Face Leg Activity Cry and Consolability (FLACC) pain scale and recordings of the baby's cries were taken in the video format. A machine-learning-based system was created to detect infant cries and pain levels. Spectrograms with the Short-Time Fourier Transform were used to convert the audio data into a time-frequency representation. Facial features combined with voice features extracted by using the Deep Learning Autoencoders was used for the classification of infant pain levels. Two types of autoencoders: Convolutional Autoencoder and Variational Autoencoder were used for both faces and voices. Result: The goal of the autoencoder was to produce a latent-vector with much smaller dimensions that was still able to recreate the data with minor losses. From the latent-vectors, a multimodal data representation for Convolutional Neural Network (CNN) was used for producing a relatively high F1 score, higher than single data modal such as the voice or facial expressions alone. Two major parts of the experiment were: 1. Building the three autoencoder models, which were autoencoder for the infant’s face, amplitude spectrogram, and dB-scaled spectrogram of infant’s voices. 2. Utilising the latent-vector result from the autoencoders to build the cry detection and pain classification models. Conclusion: In this paper, four pain classifier models with a relatively good F1 score was developed. These models were combined by using ensemble methods to improve performance, which resulted in a better F1 score.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Ensemble of multimodal deep learning autoencoder for infant cry and pain detection

Kristian¹,

Simogiarto²,

Sampurna

et al. 2023

F1000Res

View full text Add to dashboard Cite

show abstract

“…Pada penelitian yang selanjutnya, diterbitkan sebuah makalah mengenai deteksi tangis pada wajah bayi [9]. Pada penelitian ini dikenalkan fitur geometrik yang didapatkan dengan menggunakan Active Shape Model (ASM) untuk mendeteksi tangis pada wajah bayi.…”

Section: Mendesain Sebuah Deep Convolution Neural Networkunclassified

Klasifikasi Nyeri pada Video Ekspresi Wajah Bayi Menggunakan DCNN Autoencoder dan LSTM

Kristian¹,

Purnama²,

Sutanto³

et al. 2018

Jurnal Nasional Teknik Elektro dan Teknologi Informasi (JNTETI)

Self Cite

View full text Add to dashboard Cite

Babies are still unable to inform the pain they experience, therefore, babies cry when experiencing pain. With the rapid development of computer vision technologies, in the last few years, many researchers have tried to recognize pain from babies expressions using machine learning and image processing. In this paper, a research using Deep Convolution Neural Network (DCNN) Autoencoder and Long-Short Term Memory (LSTM) Network is conducted to detect cry and pain level from baby facial expression on video. DCNN Autoencoder is used to extract latent features from a single frame of baby face. Sequences of extracted latent features are then fed to LSTM so the pain level and cry can be recognized. Face detection and face landmark detection is also used to frontalize baby facial image before it is processed by DCNN Autoencoder. From the testing on DCNN autoencoder, the result shows that the best architecture used three convolutional layers and three transposed convolutional layers. As for the LSTM classifier, the best model is using four frame sequences. Intisari-Bayi belum dapat menginformasikan rasa nyeri yang dialami, karena itu bayi menangis saat mengalami nyeri. Dengan semakin berkembangnya teknologi visi komputer, beberapa tahun terakhir muncul beberapa penelitian yang mencoba mengenali nyeri pada tangis bayi memanfaatkan machine learning dan pengolahan citra. Dalam makalah ini diteliti pemanfaatan Deep Convolution Neural Network (DCNN) Autoencoder dan Long-Short Term Memory (LSTM) Network untuk deteksi tangis dan tingkat nyeri pada video wajah bayi. DCNN Autoencoder berguna untuk melakukan ekstraksi latent feature dari satu frame wajah bayi. Deretan latent feature ini kemudian diumpankan ke LSTM untuk dikenali tangis dan tingkat nyerinya. Selain itu, digunakan juga teknik face detection dan face landmark detection untuk meluruskan/menegakkan wajah bayi sebelum diproses oleh DCNN autoencoder. Dari pengujian DCNN autoencoder, didapatkan hasil terbaik dengan menggunakan tiga convolutional layer dan tiga transposed convolutional layer. Sedangkan untuk LSTM classifier, model terbaik didapatkan dalam percobaan dengan empat runtun frame.

show abstract

AICDS: An Infant Crying Detection System Based on Lightweight Convolutional Neural Network

Zhang¹,

Zou²,

Liu³

2018

Artificial Intelligence and Mobile Services – AIMS 2018

View full text Add to dashboard Cite

Ideal Modified Adachi Chaotic Neural Networks and active shape model for infant facial cry detection on still image

Cited by 4 publications

References 20 publications

Ensemble of multimodal deep learning autoencoder for infant cry and pain detection

Ensemble of multimodal deep learning autoencoder for infant cry and pain detection

Klasifikasi Nyeri pada Video Ekspresi Wajah Bayi Menggunakan DCNN Autoencoder dan LSTM

AICDS: An Infant Crying Detection System Based on Lightweight Convolutional Neural Network

Contact Info

Product

Resources

About