Techniques for Missing Value Recovering in Imbalanced Databases: Application in a Marketing Database with Massive Missing Data

Zárate, Luis Enrique; Nogueira, Bruno; Santos, T.R.A.; Song, Mark A. J.

doi:10.1109/icsmc.2006.385265

Cited by 5 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, some authors use missingness of data as a feature [48,49,7] that can be used to define phenotypes. But more often researches focus on imputation schemes, or methods for interpolate missing values [50,51,21,52-54]. And finally, some phenotyping methods just use essentially raw, unaltered EHR data [55,19,56] with the assumption that the models are flexible enough to manage and model the data complexities automatically.…”

Section: Introductionmentioning

confidence: 99%

Estimating summary statistics for electronic health record laboratory data for use in high-throughput phenotyping algorithms

Albers

Elhadad

Claassen

et al. 2018

Journal of Biomedical Informatics

View full text Add to dashboard Cite

We study the question of how to represent or summarize raw laboratory data taken from an electronic health record (EHR) using parametric model selection to reduce or cope with biases induced through clinical care. It has been previously demonstrated that the health care process (Hripcsak and Albers, 2012, 2013), as defined by measurement context (Hripcsak and Albers, 2013; Albers et al., 2012) and measurement patterns (Albers and Hripcsak, 2010, 2012), can influence how EHR data are distributed statistically (Kohane and Weber, 2013; Pivovarov et al., 2014). We construct an algorithm, PopKLD, which is based on information criterion model selection (Burnham and Anderson, 2002; Claeskens and Hjort, 2008), is intended to reduce and cope with health care process biases and to produce an intuitively understandable continuous summary. The PopKLD algorithm can be automated and is designed to be applicable in high-throughput settings; for example, the output of the PopKLD algorithm can be used as input for phenotyping algorithms. Moreover, we develop the PopKLD-CAT algorithm that transforms the continuous PopKLD summary into a categorical summary useful for applications that require categorical data such as topic modeling. We evaluate our methodology in two ways. First, we apply the method to laboratory data collected in two different health care contexts, primary versus intensive care. We show that the PopKLD preserves known physiologic features in the data that are lost when summarizing the data using more common laboratory data summaries such as mean and standard deviation. Second, for three disease-laboratory measurement pairs, we perform a phenotyping task: we use the PopKLD and PopKLD-CAT algorithms to define high and low values of the laboratory variable that are used for defining a disease state. We then compare the relationship between the PopKLD-CAT summary disease predictions and the same predictions using empirically estimated mean and standard deviation to a gold standard generated by clinical review of patient records. We find that the PopKLD laboratory data summary is substantially better at predicting disease state. The PopKLD or PopKLD-CAT algorithms are not meant to be used as phenotyping algorithms, but we use the phenotyping task to show what information can be gained when using a more informative laboratory data summary. In the process of evaluation our method we show that the different clinical contexts and laboratory measurements necessitate different statistical summaries. Similarly, leveraging the principle of maximum entropy we argue that while some laboratory data only have sufficient information to estimate a mean and standard deviation, other laboratory data captured in an EHR contain substantially more information than can be captured in higher-parameter models.

show abstract

Section: Introductionmentioning

confidence: 99%

Estimating summary statistics for electronic health record laboratory data for use in high-throughput phenotyping algorithms

Albers

Elhadad

Claassen

et al. 2018

Journal of Biomedical Informatics

View full text Add to dashboard Cite

show abstract

“…Indeed the new knowledge produced from reused and shared ontologies is still very limited (Guarino, 1998) (Blanco et al, 2008) (Coulet et al, 2008) (Sharma and Osei-Bryson, 2008) (Cardoso and Lytras, 2009). To the best of our knowledge, in spite of successful ontology approaches to solve some KDD related problems, such as, algorithms optimization (Kopanas et al, 2002) (Nogueira et al, 2007), data pre-processing tasks definition (Bouquet et al, 2002) (Zairate et al, 2006) or data mining evaluation models (Cannataro and Comito, 2003) (Brezany et al, 2008), the research to the ontological KDD process assistance is sparse and spare. Moreover, mostly of the ontology development focusing the KDD area focuses only a part of the problem, intending only to modulate data tasks (Borges et al, 2009), algorithms (Nigro et al, 2008, or evaluation models (Euler and Scholz, 2004) (Domingues and Rezende, 2005).…”

Section: Motivationmentioning

confidence: 99%

Database Marketing Process Supported by Ontologies: a Data Mining System Architecture Proposal

Pinto¹,

Guarda²

2011

New Fundamental Technologies in Data Mining

View full text Add to dashboard Cite

Ontology-supported database marketing

Pinto

Marques²,

Santos³

2009

J Database Mark Cust Strategy Manag

View full text Add to dashboard Cite

Database marketing (DBM) provides in -depth analysis of marketing databases. Knowledge discovery in database techniques is one of the most prominent approaches to supporting some of the DBM process phases. However, in many cases, the benefi ts of these tools are not fully exploited by marketers. Complexity and amount of data constitute two major factors limiting the application of knowledge discovery techniques in marketing activities. Currently, ontologies may here play an important role in the marketing discipline. Motivated by its success in the area of artifi cial intelligence, we propose an ontology-supported DBM approach. The approach aims to enhance DBM with ontology by providing detailed step-phase specifi c information. Our research work has its foundations in a double methodological approach using the Delphi and Action Research methodologies. First, we use Delphi to structure related DBM knowledge, and then we align our work to the Action Research methodology in order to systematise the knowledge extraction process and knowledge base creation. The issues raised in this paper both respond and contribute to calls for a DBM process improvement. Our work was evaluated in the relationship marketing domain focusing on a relational marketing programme database. The fi ndings of this study not only advance the state of DBM research, but also shed light on future research directions.

show abstract

Techniques for Missing Value Recovering in Imbalanced Databases: Application in a Marketing Database with Massive Missing Data

Cited by 5 publications

References 13 publications

Estimating summary statistics for electronic health record laboratory data for use in high-throughput phenotyping algorithms

Estimating summary statistics for electronic health record laboratory data for use in high-throughput phenotyping algorithms

Database Marketing Process Supported by Ontologies: a Data Mining System Architecture Proposal

Ontology-supported database marketing

Contact Info

Product

Resources

About