A linear programming-based framework for handling missing data in multi-granular data warehouses

Dimensional Data KNN-Based Imputation

“…For the incremental weight, we consider the weight of the highest-granularity as one portion and it increases by one portion for each neighboring lower-granularity level. The total weight should be equal to 1, thus the incremental hierarchy level weight of the lth level at H 2 is calculated as (4).…”

“…Data imputation is the process of replacing the missing values by some plausible values based on information available in the data [12]. The current DW data imputation research mainly focuses on factual data [25,21,4]. Yet the dimensional missing data make aggregated data incomplete and make it hard to analyse them with respect to hierarchy levels.…”

ERIS: Efficiently Measuring Discord in Multidimensional Sources

“…On the other hand, looking now at JHU tables, we aggregate regional data in three different ways: deaths per country and day, also deaths per region and week, and finally cases per region and week with a lag of three weeks (we will empirically justify this concrete value later). Under the assumption of case-fatality ratio of 1.5% (observed median on June 22nd, 2021 is 1.7% according to JHU 9 ), such transformation is applied to the cases before merging and coalescing the weekly regional cases and deaths. Daily deaths reported per country and those obtained after aggregating regions are also coalesced and then aggregated per week.…”

“…From the perspective of incompleteness in multidimensional databases, attention is paid to missing values in the measures. [37] presents an approach to maximize entropy, and [9] a linear programming-based framework to impute missing values under constraints generated by sibling data at the same aggregation level, and parent data in higher levels.…”

Internal Data Imputation in Data Warehouse Dimensions

“…Data imputation is the process of filling in missing data by plausible values based on information available in the dataset [5]. Imputation of missing data focuses on factual data, with statistic-based [11], K-Nearest Neighbour (KNN)based [3], linear programming-based [2] and hybrid (KNN and constraint programming) [1] methods. There is no research about dimensional missing data.…”