A Data-based Method for Harmonising Heterogeneous Data Modelling Techniques Across Data Mining Applications

Mwitondi, Kassim; Said, Raed A.

doi:10.12785/jsap/020312

Cited by 11 publications

(15 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The foregoing objectives are fairly standard and, conventionally, they have been pursued based on pre-defined ontologies with inherently highly dynamic parameters. Such an approach tends to randomise, not only the training and testing datasets, but also, as reported in Mwitondi and Said [9], the predictive power of classifiers. Standard solutions, in many applications, focus on competing models, voting and cross-validation-implying that results are conditional on the classifiers' internal mechanics.…”

Section: To Highlight the Potential Role Of Interdisciplinarity In Inmentioning

confidence: 99%

A robust domain partitioning intrusion detection method

Mwitondi

Said

Zargari

2019

Journal of Information Security and Applications

Self Cite

View full text Add to dashboard Cite

The capacity for data mining algorithms to learn rules from data is influenced by, inter-alia, the random nature of training and test data as well as by the diversity of domain partitioning models. Isolating normal from malicious data traffic across networks is one regular task that is naturally affected by that randomness and diversity. We propose a robust algorithm Sample-Measure-Assess (SMA) that detects intrusion based on rules learnt from multiple samples. We adapt data obtained from a set of simulations, capturing data attributes identifiable by number of bytes, destination and source of packets, protocol and nature of data flows (normal and abnormal) as well IP addresses. A fixed sample of 82,332 observations on 27 variables was drawn from a superset of 2.54 million observations on 49 variables and multiple samples were then repeatedly extracted from the former and used to train and test multiple versions of classifiers, via the algorithm. With two class labels-binary and multi-class, the dataset presents a classic example of masked and spurious groupings, making an ideal case for concept learning. The algorithm learns a model for the underlying distributions of the samples and it provides mechanics for model assessment. The settings account for our method's novelty-i.e., ability to learn concept rules from highly masked to highly spurious cases while observing model robustness. A comparative analysis of Random Forests and individually grown trees show that we can circumvent the former's dependence on multicollinearity of the trees and their individual strength in the forest by proceeding from dimensional reduction to classification using individual trees. Given data of similar structure, the algorithm can order the models in terms of optimality which, means our work can contribute towards understanding the concept of normal and malicious flows across tools. The algorithm yields results that are less sensitive to violated distributional assumptions and, hence, it yields robust parameters and provides a generalisation that can be monitored and adapted to specific low levels of variability. We discuss its potential for deployment with other classifiers and potential for extension into other applications, simply by adapting the objectives to specific conditions.

show abstract

Section: To Highlight the Potential Role Of Interdisciplinarity In Inmentioning

confidence: 99%

A robust domain partitioning intrusion detection method

Mwitondi

Said

Zargari

2019

Journal of Information Security and Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the active data, we explore various values of the methods' smoothing parameter to provide different sizes of the neighbourhood to minimise the effect of different types of randomness on the results [19,20]. With Loess-Local Regression, we fit multiple regressions in local neighborhoods of the radon flow from the active detector.…”

Section: Implementation Strategymentioning

confidence: 99%

Statistical Estimate of Radon Concentration from Passive and Active Detectors in Doha

Mwitondi

Sadig

Hassona

et al. 2018

Data

View full text Add to dashboard Cite

Abstract:Harnessing knowledge on the physical and natural conditions that affect our health, general livelihood and sustainability has long been at the core of scientific research. Health risks of ionising radiation from exposure to radon and radon decay products in homes, work and other public places entail developing novel approaches to modelling occurrence of the gas and its decaying products, in order to cope with the physical and natural dynamics in human habitats. Various data modelling approaches and techniques have been developed and applied to identify potential relationships among individual local meteorological parameters with a potential impact on radon concentrations-i.e., temperature, barometric pressure and relative humidity. In this first research work on radon concentrations in the State of Qatar, we present a combination of exploratory, visualisation and algorithmic estimation methods to try and understand the radon variations in and around the city of Doha. Data were obtained from the Central Radiation Laboratories (CRL) in Doha, gathered from 36 passive radon detectors deployed in various schools, residential and work places in and around Doha as well as from one active radon detector located at the CRL. Our key findings show high variations mainly attributable to technical variations in data gathering, as the equipment and devices appear to heavily influence the levels of radon detected. A parameter maximisation method applied to simulate data with similar behaviour to the data from the passive detectors in four of the neighbourhoods appears appropriate for estimating parameters in cases of data limitation. Data from the active detector exhibit interesting seasonal variations-with data clustering exhibiting two clearly separable groups, with passive and active detectors exhibiting a huge disagreement in readings. These patterns highlight challenges related to detection methods-in particular ensuring that deployed detectors and calculations of radon concentrations are adapted to local conditions. The study doesn't dwell much on building materials and makes rather fundamental assumptions, including an equal exhalation rate of radon from the soil across neighbourhoods, based on Doha's homogeneous underlying geological formation. The study also highlights potential extensions into the broader category of pollutants such as hydrocarbon, air particulate carbon monoxide and nitrogen dioxide at specific time periods of the year and particularly how they may tie in with global health institutions' requirements.

show abstract

“…Recent spatio-temporal variations as in [10] provide good insights into spatio-temporal variations through simulations. The foregoing research objectives allude to an interdisciplinary approach to air quality modelling and to the general research philosophy that interdisciplinary formalisation of multifaceted environmental-related data, analytical methods and procedures potentially yields consistent, comprehensive, robust and veracious results as they help minimise the effect of data randomness [11]. The paper presents a comparative statistical downscaling framework based on the core ideas of dimensional reduction as described in the following exposition.…”

Section: General: To Develop a General Framework For Training And Tesmentioning

confidence: 99%

A statistical downscaling framework for environmental mapping

et al. 2018

Self Cite

View full text Add to dashboard Cite

In recent years, knowledge extraction from data has become increasingly popular, with many numerical forecasting models, mainly falling into two major categorieschemical transport models (CTMs) and conventional statistical methods. However, due to data and model variability, data-driven knowledge extraction from highdimensional, multifaceted data in such applications require generalisations of global to regional or local conditions. Typically, generalisation is achieved via mapping global conditions to local ecosystems and human habitats which amounts to tracking and monitoring environmental dynamics in various geographical areas and their regional and global implications on human livelihood. Statistical downscaling techniques have been widely used to extract high-resolution information from regional-scale variables produced by CTMs in climate model. Conventional applications of these methods are predominantly dimensional reduction in nature, designed to reduce spatial dimension of gridded model outputs without loss of essential spatial information. Their downside is twofold-complete dependence on unlabelled design matrix and reliance on underlying distributional assumptions. We propose a novel statistical downscaling framework for dealing with data and model variability. Its power derives from training and testing multiple models on multiple samples, narrowing down global environmental phenomena to regional discordance through dimensional reduction and visualisation. Hourly ground-level ozone observations were obtained from various environmental stations maintained by the US Environmental Protection Agency, covering the summer period (June-August 2005). Regional patterns of ozone are related to local observations via repeated runs and performance assessment of multiple versions of empirical orthogonal functions or principal components and principal fitted com

show abstract

A Data-based Method for Harmonising Heterogeneous Data Modelling Techniques Across Data Mining Applications

Cited by 11 publications

References 17 publications

A robust domain partitioning intrusion detection method

A robust domain partitioning intrusion detection method

Statistical Estimate of Radon Concentration from Passive and Active Detectors in Doha

A statistical downscaling framework for environmental mapping

Contact Info

Product

Resources

About