Mining and visualising contradictory data

Abstract

A noisy dataset can contain contradictory data. Contradictory data is synonymous to incorrect data and it is important that such data be investigated and evaluated when analysing a noisy dataset. Different approaches to dealing with contradictory data have been proposed by different researchers. For example [1, 2] proposed methods for identifying and removing contradictory data in noisy datasets. However, the removal of contradictory data from a noisy dataset will increase the incompleteness in the dataset thereby reducing the soundness of any information from such set of data. It is therefore important to identify and evaluate contradictory instances when analysing a large and noisy dataset. This will improve the soundness of the analysis from such a dataset. Evidently, the analysis of big data is identified as the next frontier for innovation and advancement of technology [3, 4]. There is therefore the need to identify appropriate approaches to dealing with contradictions in a large and noisy dataset. There are different forms of contradictions. For example, there are contradictions from the use of modal words, structural, subtle lexical contrasts, as well as world knowledge

Visual Identification of Inconsistency in Pattern

“…Contradictions on the other hand, differ from one dataset to another, depending on the semantic definition of the data in the dataset. Interestingly, there are dedicated Applications such as CUBIST [19], ConTra [20], and R Package VIM [21] which enables the visualisation of the amount or pattern of contradiction and missingness in a noisy dataset. Inconsistent data whose pattern involves mutually exclusive type of contradictions is depicted by ConTra.…”

Cross-domain graph based similarity measurement of workflows

“…For instance, spectral methods are not fully able to cope with larger amounts of noise. This stems from the fact that the Eigen decomposition is very sensitive to structural errors, such as missing or spurious data known as contradictory data [7]. To address this issue, the present paper uses a technique based on Point-wise Mutual Information (PMI) extraction like [30] and discovers knowledge from provenance graphs like [31].…”

“…Today, scientific workflows have become the workhorse of Big Data analytics for scientists [5]. Aggregating these workflows with other workflows in other domains may cause challenges of Big Data [6][7][8]. The problem of enormous data of workflow is not only arises of the context of workflow but also arises of the order of its context.…”

A Machine Learning Based Framework for Predicting Student’s Academic Performance

“…In order to achieve this, they leveraged on the technological advancement of data mining to extract information from large educational data repositories [1]. When mining is introduced in educational environment, it is regarded as Educational Data Mining (EDM), which undoubtedly has become the useful way of discovering hidden information from very large educational databases.…”

“…Such information before the innovation of data mining is not utilized in decision making process. Data mining tools have brought in, its usefulness in analyzing student's trends and behaviours towards education thereby removing the use of intuition by decision makers in decision making process [1,2]. There is need for institution managers to expedite action in utilizing the vast amount of dataset of learners and other stakeholders in academic environment for multipurpose decision making.…”