Neural Architecture Search for Time Series Classification

Rakhshani, Hojjat; Fawaz, Hassan Ismail; Idoumghar, Lhassane; Forestier, Germain; Lepagnot, Julien; Weber, Jonathan; Brevilliers, Mathieu; Müller, Pierre-Alain

doi:10.1109/ijcnn48605.2020.9206721

Cited by 18 publications

(4 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other investigations of deep learning for time series classification include transfer learning [51], data augmentation [52], adversarial attacks [53], and neural architecture search [54].…”

Section: Markov Transition Field a Raw Time Series Is Discretized Wit...mentioning

confidence: 99%

Time Series Classification: A Review of Algorithms and Implementations

Faouzi

2024

Time Series Analysis - Recent Advances, New Perspectives and Applications

View full text Add to dashboard Cite

Time series classification is a subfield of machine learning with numerous real-life applications. Due to the temporal structure of the input data, standard machine learning algorithms are usually not well suited to work on raw time series. Over the last decades, many algorithms have been proposed to improve the predictive performance and the scalability of state-of-the-art models. Many approaches have been investigated, ranging from deriving new metrics to developing bag-of-words models to imaging time series to artificial neural networks. In this review, we present in detail the major contributions made to this field and mention their most prominent extensions. We dedicate a section to each category of algorithms, with an intuitive introduction on the general approach, detailed theoretical descriptions and explicit illustrations of the major contributions, and mentions of their most prominent extensions. At last, we dedicate a section to publicly available resources, namely data sets and open-source software, for time series classification. A particular emphasis is made on enumerating the availability of the mentioned algorithms in the most popular libraries. The combination of theoretical and practical contents provided in this review will help the readers to easily get started on their own work on time series classification, whether it be theoretical or practical.

show abstract

“…Other investigations of deep learning for time series classification include transfer learning [51], data augmentation [52], adversarial attacks [53], and neural architecture search [54].…”

Section: Markov Transition Field a Raw Time Series Is Discretized Wit...mentioning

confidence: 99%

Time Series Classification: A Review of Algorithms and Implementations

Faouzi

2024

Time Series Analysis - Recent Advances, New Perspectives and Applications

View full text Add to dashboard Cite

show abstract

“…Recently, NAS methods have also been applied to imagebased and skeleton-based activity recognition (Popescu et al, 2020;Zhang et al, 2020), as well as domain-agnostic timeseries classification (Rakhshani et al, 2020), with promising results. This work represents the first exploration of NAS with performance prediction for wearable sensor-based HAR.…”

Section: Neural Architecture Searchmentioning

confidence: 99%

Speeding up deep neural architecture search for wearable activity recognition with early prediction of converged performance

Pellatt

Roggen²

2022

Front. Comput. Sci.

View full text Add to dashboard Cite

Neural architecture search (NAS) has the potential to uncover more performant networks for human activity recognition from wearable sensor data. However, a naive evaluation of the search space is computationally expensive. We introduce neural regression methods for predicting the converged performance of a deep neural network (DNN) using validation performance in early epochs and topological and computational statistics. Our approach shows a significant improvement in predicting converged testing performance over a naive approach taking the ranking of the DNNs at an early epoch as an indication of their ranking on convergence. We apply this to the optimization of the convolutional feature extractor of an LSTM recurrent network using NAS with deep Q-learning, optimizing the kernel size, number of kernels, number of layers, and the connections between layers, allowing for arbitrary skip connections and dimensionality reduction with pooling layers. We find architectures which achieve up to 4% better F1 score on the recognition of gestures in the Opportunity dataset than our implementation of DeepConvLSTM and 0.8% better F1 score than our implementation of state-of-the-art model Attend and Discriminate, while reducing the search time by more than 90% over a random search. This opens the way to rapidly search for well-performing dataset-specific architectures. We describe the computational implementation of the system (software frameworks, computing resources) to enable replication of this work. Finally, we lay out several future research directions for NAS which the community may pursue to address ongoing challenges in human activity recognition, such as optimizing architectures to minimize power, minimize sensor usage, or minimize training data needs.

show abstract

“…Recent research also explores the use of NAS for healthcare applications, such as electroencephalography (EEG) data processing [30], muscle fatigue detection [31], cardiac abnormality diagnosis [32], and heartbeat classification [33]. Moreover, an NAS was developed by leveraging k-fold cross-validation, and the deep learning model was evaluated on data from the UCR archive [34].…”

mentioning

confidence: 99%

AutoNet-Generated Deep Layer-Wise Convex Networks for ECG Classification

Shen,

Lu,

Zhu

et al. 2024

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

The design of neural networks typically involves trial-and-error, a time-consuming process for obtaining an optimal architecture, even for experienced researchers. Additionally, it is widely accepted that loss functions of deep neural networks are generally non-convex with respect to the parameters to be optimised. We propose the Layer-wise Convex Theorem to ensure that the loss is convex with respect to the parameters of a given layer, achieved by constraining each layer to be an overdetermined system of non-linear equations. Based on this theorem, we developed an end-to-end algorithm (the AutoNet) to automatically generate layer-wise convex networks (LCNs) for any given training set. We then demonstrate the performance of the AutoNet-generated LCNs (AutoNet-LCNs) compared to state-of-the-art models on three electrocardiogram (ECG) classification benchmark datasets, with further validation on two non-ECG benchmark datasets for more general tasks. The AutoNet-LCN was able to find networks customised for each dataset without manual fine-tuning under 2 GPU-hours, and the resulting networks outperformed the state-of-the-art models with fewer than 5% parameters on all the above five benchmark datasets. The efficiency and robustness of the AutoNet-LCN markedly reduce model discovery costs and enable efficient training of deep learning models in resource-constrained settings.

show abstract

Neural Architecture Search for Time Series Classification

Cited by 18 publications

References 39 publications

Time Series Classification: A Review of Algorithms and Implementations

Time Series Classification: A Review of Algorithms and Implementations

Speeding up deep neural architecture search for wearable activity recognition with early prediction of converged performance

AutoNet-Generated Deep Layer-Wise Convex Networks for ECG Classification

Contact Info

Product

Resources

About