Machine Learning-Based Fast Banknote Serial Number Recognition Using Knowledge Distillation and Bayesian Optimization

Choi, Eunjeong; Chae, Somi; Kim, Jeongtae

doi:10.3390/s19194218

Cited by 12 publications

(6 citation statements)

References 26 publications

(65 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The genetic algorithm was then applied out to optimize the banknote regions. More recently, a machine learning-based approach for simultaneous ROI extraction and character classification was presented [ 22 ]. Based on the use of knowledge distillation, the complexity can be reduced with a simple model for fast computation.…”

Section: Related Workmentioning

confidence: 99%

Structured Cluster Detection from Local Feature Learning for Text Region Extraction

Lin

Hsu

2023

Entropy

View full text Add to dashboard Cite

The detection of regions of interest is commonly considered as an early stage of information extraction from images. It is used to provide the contents meaningful to human perception for machine vision applications. In this work, a new technique for structured region detection based on the distillation of local image features with clustering analysis is proposed. Different from the existing methods, our approach takes the application-specific reference images for feature learning and extraction. It is able to identify text clusters under the sparsity of feature points derived from the characters. For the localization of structured regions, the cluster with high feature density is calculated and serves as a candidate for region expansion. An iterative adjustment is then performed to enlarge the ROI for complete text coverage. The experiments carried out for text region detection of invoice and banknote demonstrate the effectiveness of the proposed technique.

show abstract

Section: Related Workmentioning

confidence: 99%

Structured Cluster Detection from Local Feature Learning for Text Region Extraction

Lin

Hsu

2023

Entropy

View full text Add to dashboard Cite

show abstract

“…Park et al [ 12 ] introduced channel and spatial correlation loss and the adaptive Cross-Entropy (CE) loss for applying KD to semantic segmentation problem. Choi et al [ 13 ] investigated KD on serial number recognition task and applied the Bayesian optimization method to automatically tune KD’s hyper-parameters. Chechlinski et al [ 14 ] develop a light system for weeds and crops identification with KD.…”

Section: Introductionmentioning

confidence: 99%

Memory-Replay Knowledge Distillation

Wang

Zhang

2021

Sensors

View full text Add to dashboard Cite

Knowledge Distillation (KD), which transfers the knowledge from a teacher to a student network by penalizing their Kullback–Leibler (KL) divergence, is a widely used tool for Deep Neural Network (DNN) compression in intelligent sensor systems. Traditional KD uses pre-trained teacher, while self-KD distills its own knowledge to achieve better performance. The role of the teacher in self-KD is usually played by multi-branch peers or the identical sample with different augmentation. However, the mentioned self-KD methods above have their limitation for widespread use. The former needs to redesign the DNN for different tasks, and the latter relies on the effectiveness of the augmentation method. To avoid the limitation above, we propose a new self-KD method, Memory-replay Knowledge Distillation (MrKD), that uses the historical models as teachers. Firstly, we propose a novel self-KD training method that penalizes the KD loss between the current model’s output distributions and its backup outputs on the training trajectory. This strategy can regularize the model with its historical output distribution space to stabilize the learning. Secondly, a simple Fully Connected Network (FCN) is applied to ensemble the historical teacher’s output for a better guidance. Finally, to ensure the teacher outputs offer the right class as ground truth, we correct the teacher logit output by the Knowledge Adjustment (KA) method. Experiments on the image (dataset CIFAR-100, CIFAR-10, and CINIC-10) and audio (dataset DCASE) classification tasks show that MrKD improves single model training and working efficiently across different datasets. In contrast to the existing fancy self-KD methods with various external knowledge, the effectiveness of MrKD sheds light on the usually abandoned historical models during the training trajectory.

show abstract

“…DBNs have been developed and used in many fields, such as in acoustic modeling [16], medical classification [17] et al In reference [8], the Bayesian regularized deep belief network (R-DBN) was first proposed and applied to the extraction of coupling matrix, providing a new solution for inverse filter modeling. However, Bayesian optimization algorithms have a low number of iterations in optimizing hyperparameters and can easily fall into local optimization [18]. In reference [19], a PSO-DBN-based model is proposed to improve the accuracy and analysis efficiency of the model by adaptively adjusting the DBN model parameters through PSO.…”

Section: Introductionmentioning

confidence: 99%

Design of UWB Antenna Based on Improved Deep Belief Network and Extreme Learning Machine Surrogate Models

Nan

Xie²,

Gao

et al. 2021

IEEE Access

View full text Add to dashboard Cite

In the design of conventional microwave devices, the parameters need to be continuously optimized to meet the desired targets, and the whole process is time-consuming and laborious. As a surrogate model, machine learning is an effective optimization method. However, in the modeling process, the highdimensional data processing and the complex nonlinear relationship between parameters is a problem to be solved. This paper proposes a deep learning model for designing UWB antennas, which determines the model structure of deep belief network (DBN) by particle swarm algorithm (PSO), and then combines DBN and extreme learning machine (ELM). The proposed model can obtain higher feature learning capability and nonlinear function approximation capability, and has been applied to the optimal design of the whole structure of the fractal antenna and the notch structure of the MIMO antenna, and its S-parameters are well fitted while meeting the requirements of the design targets. The DBN-ELM method obtains the good results when compared with common modeling methods using the same training samples (the root mean square error tested is 11.87% in the fractal antenna and 3.56% in the MIMO antenna). Overall, the proposed DBN-ELM model has higher predictive and generalization capabilities, which can also be used to model more complex antenna structures. INDEX TERMSDeep belief network (DBN),Extreme learning machine (ELM),Particle swarm optimization (PSO),UWB antenna.

show abstract

Machine Learning-Based Fast Banknote Serial Number Recognition Using Knowledge Distillation and Bayesian Optimization

Cited by 12 publications

References 26 publications

Structured Cluster Detection from Local Feature Learning for Text Region Extraction

Structured Cluster Detection from Local Feature Learning for Text Region Extraction

Memory-Replay Knowledge Distillation

Design of UWB Antenna Based on Improved Deep Belief Network and Extreme Learning Machine Surrogate Models

Contact Info

Product

Resources

About