Scaling Genetic Programming to Large Datasets Using Hierarchical Dynamic Subset Selection

Curry, Robert H.; Lichodzijewski, Peter; Heywood, Malcolm I.

doi:10.1109/tsmcb.2007.896406

Cited by 43 publications

(29 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the convergence of the algorithm is faster when GA is employed. Subset generation [48] is a method of heuristic search, in which each instance in the search space specifies a candidate solution for subset evaluation. The decision process of this method is determined by some basic issues.…”

Section: Genetic-based Feature Selectionmentioning

confidence: 99%

Intelligent feature selection and classification techniques for intrusion detection in networks: a survey

Ganapathy

Kanagasabai

Muthurajkumar

et al. 2013

J Wireless Com Network

169

View full text Add to dashboard Cite

Rapid growth in the Internet usage and diverse military applications have led researchers to think of intelligent systems that can assist the users and applications in getting the services by delivering required quality of service in networks. Some kinds of intelligent techniques are appropriate for providing security in communication pertaining to distributed environments such as mobile computing, e-commerce, telecommunication, and network management. In this paper, a survey on intelligent techniques for feature selection and classification for intrusion detection in networks based on intelligent software agents, neural networks, genetic algorithms, neuro-genetic algorithms, fuzzy techniques, rough sets, and particle swarm intelligence has been proposed. These techniques have been useful for effectively identifying and preventing network intrusions in order to provide security to the Internet and to enhance the quality of service. In addition to the survey on existing intelligent techniques for intrusion detection systems, two new algorithms namely intelligent rule-based attribute selection algorithm for effective feature selection and intelligent rule-based enhanced multiclass support vector machine have been proposed in this paper.Keywords: Survey; Intrusion detection system; Neural networks; Fuzzy systems; Swarm intelligence; Particle swarm intelligence Review Intrusion detection systemsRecently, Internet has become a part and parcel of daily life. The current internet-based information processing systems are prone to different kinds of threats which lead to various types of damages resulting in significant losses. Therefore, the importance of information security is evolving quickly. The most basic goal of information security is to develop defensive information systems which are secure from unauthorized access, use, disclosure, disruption, modification, or destruction. Moreover, information security minimizes the risks related to the three main security goals namely confidentiality, integrity, and availability.Various systems have been designed in the past to identify and block the Internet-based attacks. The most important systems among them are intrusion detection systems (IDS) since they resist external attacks effectively. Moreover, IDSs provide a wall of defense which overcomes the attack of computer systems on the Internet. IDS could be used to detect different types of attacks on network communications and computer system usage where the traditional firewall cannot perform well. Intrusion detection is based on an assumption that the behavior of intruders differ from a legal user [1]. Generally, IDSs are broadly classified into two categories namely anomaly and misuse detection systems based on their detection approaches [2,3]. Anomaly intrusion detection determines whether deviation from the established normal usage patterns can be flagged as intrusions. On the other hand, misuse detection systems detect the violations of permissions effectively. Intrusion detection systems can be built by u...

show abstract

Section: Genetic-based Feature Selectionmentioning

confidence: 99%

Intelligent feature selection and classification techniques for intrusion detection in networks: a survey

Ganapathy

Kanagasabai

Muthurajkumar

et al. 2013

J Wireless Com Network

169

View full text Add to dashboard Cite

show abstract

“…This problem is related to both the number and size of the training data. In the case of machine learning, research on intelligent sampling methods [34,67] might be necessary to enable the use of the methods. This problem has been already explored in other contexts, such as selecting a subset of instances for the user to label or reducing data sets for evolutionary algorithms fitness computation, and some such wellunderstood methods could certainly be applied.…”

Section: Challenges Relating To Datasets and Trainingmentioning

confidence: 99%

Contrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms

Pappa

Ochoa

Hyde

et al. 2013

Genet Program Evolvable Mach

View full text Add to dashboard Cite

The fields of machine meta-learning and hyper-heuristic optimisation have developed mostly independently of each other, although evolutionary algorithms (particularly genetic programming) have recently played an important role in the development of both fields. Recent work in both fields shares a common goal, that of automating as much of the algorithm design process as possible. In this paper we first provide a historical perspective on automated algorithm design, and then we discuss similarities and differences between meta-learning in the field of supervised machine learning (classification) and hyper-heuristics in the field of optimisation. This discussion focuses on the dimensions of the problem space, the algorithm space and the performance measure, as well as clarifying important issues related to different levels of automation and generality in both fields. We also discuss important research directions, challenges and foundational issues in meta-learning and hyper-heuristic research. It is important to emphasize that this paper is not a survey, as several surveys on the areas of meta-learning and hyper-heuristics (separately) have been previously published. The main contribution of the paper is to contrast meta-learning and hyper-heuristics methods and concepts, in order to 123Genet Program Evolvable Mach DOI 10.1007/s10710-013-9186-9 promote awareness and cross-fertilisation of ideas across the (by and large, nonoverlapping) different communities of meta-learning and hyper-heuristic researchers. We hope that this cross-fertilisation of ideas can inspire interesting new research in both fields and in the new emerging research area which consists of integrating those fields.

show abstract

“…To make sure that all instances are eventually seen by the system, selection based on age is occasionally performed. In order to extend this paradigm to datasets that cannot be entirely cached in memory, hierarchical versions of DSS have also been proposed [30,31]. These algorithms first partition the dataset into blocks that fit entirely in memory then limit subset selection to blocks that have already been cached in order to take advantage of faster read speeds.…”

Section: Genetic Programming On Large Classification Problemsmentioning

confidence: 99%

“…These algorithms first partition the dataset into blocks that fit entirely in memory then limit subset selection to blocks that have already been cached in order to take advantage of faster read speeds. The balanced block version of the algorithm [30] further refines selection to handle highly unbalanced class distributions.…”

Section: Genetic Programming On Large Classification Problemsmentioning

confidence: 99%

Coevolutionary bid-based genetic programming for problem decomposition in classification

Lichodzijewski

Heywood

2008

Genet Program Evolvable Mach

View full text Add to dashboard Cite

In this work a cooperative, bid-based, model for problem decomposition is proposed with application to discrete action domains such as classification. This represents a significant departure from models where each individual constructs a direct input-outcome map, for example, from the set of exemplars to the set of class labels as is typical under the classification domain. In contrast, the proposed model focuses on learning a bidding strategy based on the exemplar feature vectors; each individual is associated with a single discrete action and the individual with the maximum bid 'wins' the right to suggest its action. Thus, the number of individuals associated with each action is a function of the intra-action bidding behaviour. Credit assignment is designed to reward correct but unique bidding strategies relative to the target actions. An advantage of the model over other teaming methods is its ability to automatically determine the number of and interaction between cooperative team members. The resulting model shares several traits with learning classifier systems and as such both approaches are benchmarked on nine large classification problems. Moreover, both of the evolutionary models are compared against the deterministic Support Vector Machine classification algorithm. Performance assessment considers the computational, classification, and complexity characteristics of the resulting solutions. The bid-based model is found to provide simple yet effective solutions that are robust to wide variations in the class representation. Support Vector Machines and classifier systems tend to perform better under balanced datasets albeit resulting in black-box solutions.

show abstract

Scaling Genetic Programming to Large Datasets Using Hierarchical Dynamic Subset Selection

Cited by 43 publications

References 12 publications

Intelligent feature selection and classification techniques for intrusion detection in networks: a survey

Intelligent feature selection and classification techniques for intrusion detection in networks: a survey

Contrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms

Coevolutionary bid-based genetic programming for problem decomposition in classification

Contact Info

Product

Resources

About