Missing data management and statistical measurement of socio-economic status: application of big data

Wubetie, Habtamu Tilaye

doi:10.1186/s40537-017-0099-y

Cited by 23 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The RITS toolbox allows for missing data by linearly interpolating missing values. This imputation may lead to a small bias in parameter estimates in comparison to other methods [ 22 , 23 ].We therefore recommend against analyzing series with missing data in RITS. In addition, RITS assumes that the time series are recorded for the same time points across units and analyzes data only for the overlapping time points.…”

Section: Discussionmentioning

confidence: 99%

RITS: a toolbox for assessing complex interventions via interrupted time series models

Cruz

Pinto-Orellana

Gillen

et al. 2021

BMC Med Res Methodol

View full text Add to dashboard Cite

Background Various interacting and interdependent components comprise complex interventions. These components create difficulty in assessing the true impact of interventions designed to improve patient-centered outcomes. Interrupted time series (ITS) designs borrow from case-crossover designs and serve as quasi-experimental methodology able to retrospectively assess the impact of an intervention while accounting for temporal correlation. While ITS designs are aptly situated for studying the impacts of large-scale public health policies, existing ITS software implement rigid ITS methodology that often assume the pre- and post-intervention phases are fully differentiated (by a known change-point or set of time points) and do not allow for changes in both the mean functions and correlation structure. Results This article describes the Robust Interrupted Time Series (RITS) toolbox, a stand-alone user-friendly application researchers can use to implement flexible ITS models that estimate the lagged effect of an intervention on an outcome, level and trend changes, and post-intervention changes in the correlation structure, for single and multiple ITS. The RITS toolbox incorporates a formal test for the existence of a change in the outcome and estimates a change-point over a set of possible change-points defined by the researcher. In settings with multiple ITS, RITS provides a global over-all units change-point and allows for unit-specific changes in the mean functions and correlation structures. Conclusions The RITS toolbox is the first piece of software that allows researchers to use flexible ITS models that test for the existence of a change-point, estimate the change-point (if estimation is desired), and allow for changes in both the mean functions and correlation structures at the change point. RITS does not require any knowledge of a statistical (or otherwise) programming language, is freely available to the community, and may be downloaded and used on a local machine to ensure data protection.

show abstract

Section: Discussionmentioning

confidence: 99%

RITS: a toolbox for assessing complex interventions via interrupted time series models

Cruz

Pinto-Orellana

Gillen

et al. 2021

BMC Med Res Methodol

View full text Add to dashboard Cite

show abstract

“…However, the success ratio depends on the number of missing values of the selected attributes. Therefore, the process of compensating the missing values in the selected attributes is necessary to increase the probability of having records that refer to the same entity of the real world in the same group [16]. • Data matching: This process is important in finding duplicate records in a data set.…”

Section: The Challenge Of Detecting Duplicatesmentioning

confidence: 99%

Missing values compensation in duplicates detection using hot deck method

2021

View full text Add to dashboard Cite

Duplicate record is a common problem within data sets especially in huge volume databases. The accuracy of duplicate detection determines the efficiency of duplicate removal process. However, duplicate detection has become more challenging due to the presence of missing values within the records where during the clustering and matching process, missing values can cause records deemed similar to be inserted into the wrong group, hence, leading to undetected duplicates. In this paper, duplicate detection improvement was proposed despite the presence of missing values within a data set through Duplicate Detection within the Incomplete Data set (DDID) method. The missing values were hypothetically added to the key attributes of three data sets under study, using an arbitrary pattern to simulate both complete and incomplete data sets. The results were analyzed, then, the performance of duplicate detection was evaluated by using the Hot Deck method to compensate for the missing values in the key attributes. It was hypothesized that by using Hot Deck, duplicate detection performance would be improved. Furthermore, the DDID performance was compared to an early duplicate detection method namely DuDe, in terms of its accuracy and speed. The findings yielded that even though the data sets were incomplete, DDID was able to offer a better accuracy and faster duplicate detection as compared to DuDe. The results of this study offer insights into constraints of duplicate detection within incomplete data sets.

show abstract

“…However, the success ratio depends on the number of missing values of the selected attributes. Therefore, the process of compensating the missing values in the selected attributes is necessary to increase the probability of having records that are likely referring to the same real-world entity in the same group [16]. • Data Matching: this process is important in finding duplicate records in a data set.…”

Section: The Challenge Of Detecting Duplicatesmentioning

confidence: 99%

“…1 Deletion: A completely missing point is skipped when utilizing a dataset with the missing value [35]. 2 Interpolation: In a geometric sense, a line is drawn between the ends of sequences on either side of the missing value [16]. 3 Imputation: This operation involves fixing of value at the point where it is missing in the string using a specific algorithm like K-means, where the missing value will be replaced with a substituted value.…”

Section: Related Workmentioning

confidence: 99%

Missing Values Compensation in Duplicates Detection Using Hot Deck

Ali

Emran

Asmai

2021

Preprint

View full text Add to dashboard Cite

Duplicate record is a known problem within the datasets especially within databases of huge volumes. The accuracy of duplicates detection determines the efficiency of the duplicates removal process. Unfortunately, the effort to detect duplicates becomes more challenging due to the presence of missing values within the records. This is because, during the clustering and matching process, missing values can cause records that are similar to be assigned in a wrong group, causing the duplicates left undetected. In this paper, we present how duplicates detection can be improved even though missing values are present within a data set using our Duplicates Detection within the Incomplete Data set (DDID) method. We hypothetically add the missing values to the key attributes of two datasets under study, using an arbitrary pattern to simulate both complete and incomplete data sets. We analyze the results to evaluate the performance of duplicates detection using the Hot Deck method to compensate for the missing values in the key attributes. We hypothesize that by using Hot Deck, there is a performance improvement in duplicates detection. The performance of the DDID is compared with an early duplicates detection method (called DuDe) in terms of its accuracy and speed. The findings of the experiment show that, even though the data sets are incomplete, DDID is capable to offer better accuracy and faster duplicates detection as compared to a benchmark method (called DuDe). The results of this study contribute to duplicates detection under incomplete data sets constraint.

show abstract

Missing data management and statistical measurement of socio-economic status: application of big data

Cited by 23 publications

References 15 publications

RITS: a toolbox for assessing complex interventions via interrupted time series models

RITS: a toolbox for assessing complex interventions via interrupted time series models

Missing values compensation in duplicates detection using hot deck method

Missing Values Compensation in Duplicates Detection Using Hot Deck

Contact Info

Product

Resources

About