Meta-Modelling Meta-Learning

Hartmann, Thomas; Moawad, Assaad; Schockaert, Cédric; Fouquet, Francois; Traon, Yves Le

doi:10.1109/models.2019.00014

Cited by 15 publications

(10 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Answering this question would pave the way for the application of model-driven engineering techniques to the detection of errors in DL programs. In [25], researchers proposed a meta-model for meta-learning. They presented an overview of the meta-learning concepts -on a meta-modelling level-with possible variabilities and discussed how their meta-model could be integrated into existing modelling frameworks and tools.…”

Section: Meta-modeling DL Programsmentioning

confidence: 99%

“…However, the paper keeps its interesting ideas in a high-level of abstraction. A meta-model for meta-learning is presented in [25], alongside a discussion of its possible integration into state-of-the-art modelling frameworks. Meta-learning or automatic machine learning is a recent approach that applies learning algorithms on metadata about machine learning experiments to automatically improve the learning process.…”

Section: Related Workmentioning

confidence: 99%

“…Although the presented meta-model has similarities with our meta-model, the scope is different: while they focused on meta-learning, in this paper, we present a meta-model of DL programs. Moreover, [25] did not explore the possibility of identifying bugs in learning programs through model verification.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Fault Detection for Deep Learning Programs Using Graph Transformations

Nikanjam

Braiek

Morovati

et al. 2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Nowadays, we are witnessing an increasing demand in both corporates and academia for exploiting Deep Learning ( DL ) to solve complex real-world problems. A DL program encodes the network structure of a desirable DL model and the process by which the model learns from the training dataset. Like any software, a DL program can be faulty, which implies substantial challenges of software quality assurance, especially in safety-critical domains. It is therefore crucial to equip DL development teams with efficient fault detection techniques and tools. In this article, we propose NeuraLint , a model-based fault detection approach for DL programs, using meta-modeling and graph transformations. First, we design a meta-model for DL programs that includes their base skeleton and fundamental properties. Then, we construct a graph-based verification process that covers 23 rules defined on top of the meta-model and implemented as graph transformations to detect faults and design inefficiencies in the generated models (i.e., instances of the meta-model). First, the proposed approach is evaluated by finding faults and design inefficiencies in 28 synthesized examples built from common problems reported in the literature. Then NeuraLint successfully finds 64 faults and design inefficiencies in 34 real-world DL programs extracted from Stack Overflow posts and GitHub repositories. The results show that NeuraLint effectively detects faults and design issues in both synthesized and real-world examples with a recall of 70.5% and a precision of 100%. Although the proposed meta-model is designed for feedforward neural networks, it can be extended to support other neural network architectures such as recurrent neural networks. Researchers can also expand our set of verification rules to cover more types of issues in DL programs.

show abstract

Section: Meta-modeling DL Programsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Fault Detection for Deep Learning Programs Using Graph Transformations

Nikanjam

Braiek

Morovati

et al. 2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…or, can we define a meta-model of DRL programs and how can we model a DRL program? In [12], researchers proposed a meta-model for meta-learning. They presented an overview of the meta-learning concepts -on a meta-modelling level-with possible variabilities and discussed how their meta-model could be integrated into existing modelling frameworks and tools.…”

Section: A Meta-model For Drl Programsmentioning

confidence: 99%

Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach

Nikanjam¹,

Morovati²,

Khomh³

et al. 2021

Preprint

View full text Add to dashboard Cite

A growing demand is witnessed in both industry and academia for employing Deep Learning (DL) in various domains to solve real-world problems. Deep Reinforcement Learning (DRL) is the application of DL in the domain of Reinforcement Learning (RL). Like any software systems, DRL applications can fail because of faults in their programs. In this paper, we present the first attempt to categorize faults occurring in DRL programs. We manually analyzed 761 artifacts of DRL programs (from Stack Overflow posts and GitHub issues) developed using well-known DRL frameworks (OpenAI Gym, Dopamine, Keras-rl, Tensorforce) and identified faults reported by developers/users. We labeled and taxonomized the identified faults through several rounds of discussions. The resulting taxonomy is validated using an online survey with 19 developers/researchers. To allow for the automatic detection of faults in DRL programs, we have defined a meta-model of DRL programs and developed DRLinter, a model-based fault detection approach that leverages static analysis and graph transformations. The execution flow of DRLinter consists in parsing a DRL program to generate a model conforming to our meta-model and applying detection rules on the model to identify faults occurrences. The effectiveness of DRLinter is evaluated using 15 synthetic DRL programs in which we injected faults observed in the analyzed artifacts of the taxonomy. The results show that DRLinter can successfully detect faults in all synthetic faulty programs.

show abstract

“…On the other hand, it can also lead to the high computational cost when exploring the suitable configurations. Comparatively, feature engineering is an efficient process which extracts the meta features from the raw dataset [16]. In this research, a feature engineering based meta learning process has been explored here.…”

Section: Literature Reviewmentioning

confidence: 99%

A Novel Meta Learning Framework for Feature Selection using Data Synthesis and Fuzzy Similarity

Shen

Chen

Garibaldi

2020

Preprint

View full text Add to dashboard Cite

This paper presents a novel meta learning framework for feature selection (FS) based on fuzzy similarity. The proposed method aims to recommend the best FS method from four candidate FS methods for any given dataset. This is achieved by firstly constructing a large training data repository using data synthesis. Six meta features that represent the characteristics of the training dataset are then extracted. The best FS method for each of the training datasets is used as the meta label. Both the meta features and the corresponding meta labels are subsequently used to train a classification model using a fuzzy similarity measure based framework. Finally the trained model is used to recommend the most suitable FS method for a given unseen dataset. This proposed method was evaluated based on eight public datasets of real-world applications. It successfully recommended the best method for five datasets and the second best method for one dataset, which outperformed any of the four individual FS methods. Besides, the proposed method is computationally efficient for algorithm selection, leading to negligible additional time for the feature selection process. Thus, the paper contributes a novel method for effectively recommending which feature selection method to use for any new given dataset.

show abstract

Meta-Modelling Meta-Learning

Cited by 15 publications

References 17 publications

Automatic Fault Detection for Deep Learning Programs Using Graph Transformations

Automatic Fault Detection for Deep Learning Programs Using Graph Transformations

Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach

A Novel Meta Learning Framework for Feature Selection using Data Synthesis and Fuzzy Similarity

Contact Info

Product

Resources

About