KDCTime: Knowledge distillation with calibration on InceptionTime for time-series classification

Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey

“…Finally, two-stage convolution and two fully connected layers are applied to classify the images generated by RP. [84] 2019 Inception V1 InceptionTime [12] 2019 Inception V4 Ensemble EEG-inception [85] 2021 InceptionTime Inception-FCN [86] 2021 InceptionTime + FCN KDCTime [87] 2022 InceptionTime Knowledge Distillation, Label smoothing computer vision domain, pre-trained Inception v3 [79] was used to map the GADF images into a 2048-dimensional vector space. In the final stage, a multilayer perceptron (MLP) is used with three hidden layers, and a softmax activation function for classification [75].…”

“…The output of the second inception block is passed to a GAP layer before feeding into a softmax classifier. Due to the favorable performance of InceptionTime for time series classification, various extensions such as EEGinception [85], InceptionFCN [86], and KDCTime [87] have been proposed. Like InceptionTime, EEG-inception uses several inception layers and residual connections as its backbone.…”

“…Also, to tackle the limitation in the training data, noise addition-based data augmentation of EEG signals is proposed, which increases the average accuracy. Inception-FCN focuses on combining two well-known deep learning techniques, namely the Inception module and the Fully Convolutional Network [86] In KDCTime [87], the authors first proposed label smoothing for InceptionTime (LSTime), which uses the soft label information instead of only hard labels. Next, instead of manually adjusting soft labels by LSTime, knowledge distillation for InceptionTime (KDTime) is proposed to automatically generate soft labels by the teacher model while compressing the inference model.…”