Heterogeneous Data and Big Data Analytics

Wang, Lidong

doi:10.12691/acis-3-1-3

Cited by 105 publications

(53 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, in practice, any collected data are effectively a sample, and may not be a true representation of the population under study. Before training the model, it is often good to check for sample bias, and where possible check for errors stemming from sample size, 8 , 9 heterogeneity, 10 noise, 11 and confounding factors. 12 …”

Section: Good Prediction Accuracy Often Does Not Imply Domain Relevanmentioning

confidence: 99%

Extensions of the External Validation for Checking Learned Model Interpretability and Generalizability

et al. 2020

View full text Add to dashboard Cite

Summary We discuss the validation of machine learning models, which is standard practice in determining model efficacy and generalizability. We argue that internal validation approaches, such as cross-validation and bootstrap, cannot guarantee the quality of a machine learning model due to potentially biased training data and the complexity of the validation procedure itself. For better evaluating the generalization ability of a learned model, we suggest leveraging on external data sources from elsewhere as validation datasets, namely external validation. Due to the lack of research attractions on external validation, especially a well-structured and comprehensive study, we discuss the necessity for external validation and propose two extensions of the external validation approach that may help reveal the true domain-relevant model from a candidate set. Moreover, we also suggest a procedure to check whether a set of validation datasets is valid and introduce statistical reference points for detecting external data problems.

show abstract

Section: Good Prediction Accuracy Often Does Not Imply Domain Relevanmentioning

confidence: 99%

Extensions of the External Validation for Checking Learned Model Interpretability and Generalizability

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Data are gained from widely heterogeneous sources, both from structured meta-data (data from information systems organized as archives) and non-structured mega-data (images, annotations, etc.) [ 17 , 18 ]. These Big-data systems continuously growing, so adequate tools and methodologies to extract and process the information are needed.…”

Section: Big Data In Biomedicinementioning

confidence: 99%

System-Wide Pollution of Biomedical Data: Consequence of the Search for Hub Genes of Hepatocellular Carcinoma Without Spatiotemporal Consideration

Sharma

Colonna

2021

Mol Diagn Ther

View full text Add to dashboard Cite

Biomedical institutions rely on data evaluation and are turning into data factories. Big-data storage centers, supercomputing systems, and increased algorithmic efficiency allow us to analyze the ever-increasing amount of data generated every day in biomedical research centers. In network science, the principal intrinsic problem is how to integrate the data and information from different experiments on genes or proteins. Data curation is an essential process in annotating new functional data to known genes or proteins, undertaken by a biobank curator, which is then reflected in the calculated networks. We provide an example of how protein–protein networks today have space-time limits. The next step is the integration of data and information from different biobanks. Omics data and networks are essential parts of this step but also have flawed protocols and errors. Consider data from patients with cancer: from biopsy procedures to experimental tests, to archiving methods and computational algorithms, these are continuously handled so require critical and continuous “updates” to obtain reproducible, reliable, and correct results. We show, as a second example, how all this distorts studies in cellular hepatocellular carcinoma. It is not unlikely that these flawed data have been polluting biobanks for some time before stringent conditions for the veracity of data were implemented in Big data. Therefore, all this could contribute to errors in future medical decisions. Supplementary Information The online version contains supplementary material available at 10.1007/s40291-020-00505-3.

show abstract

“…Heterogeneity is one of the main characteristics of big data and heterogeneous data, causing problems in the generation and analytics of change detection results [237]. From the data source perspective, RS technology can provide various data types for change detection, such as SAR, GIS data, high-resolution satellite images, and various time and space measured data.…”

Section: Heterogeneous Big Data Processingmentioning

confidence: 99%

Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges

et al. 2020

View full text Add to dashboard Cite

Change detection based on remote sensing (RS) data is an important method of detecting changes on the Earth’s surface and has a wide range of applications in urban planning, environmental monitoring, agriculture investigation, disaster assessment, and map revision. In recent years, integrated artificial intelligence (AI) technology has become a research focus in developing new change detection methods. Although some researchers claim that AI-based change detection approaches outperform traditional change detection approaches, it is not immediately obvious how and to what extent AI can improve the performance of change detection. This review focuses on the state-of-the-art methods, applications, and challenges of AI for change detection. Specifically, the implementation process of AI-based change detection is first introduced. Then, the data from different sensors used for change detection, including optical RS data, synthetic aperture radar (SAR) data, street view images, and combined heterogeneous data, are presented, and the available open datasets are also listed. The general frameworks of AI-based change detection methods are reviewed and analyzed systematically, and the unsupervised schemes used in AI-based change detection are further analyzed. Subsequently, the commonly used networks in AI for change detection are described. From a practical point of view, the application domains of AI-based change detection methods are classified based on their applicability. Finally, the major challenges and prospects of AI for change detection are discussed and delineated, including (a) heterogeneous big data processing, (b) unsupervised AI, and (c) the reliability of AI. This review will be beneficial for researchers in understanding this field.

show abstract

Heterogeneous Data and Big Data Analytics

Cited by 105 publications

References 8 publications

Extensions of the External Validation for Checking Learned Model Interpretability and Generalizability

Extensions of the External Validation for Checking Learned Model Interpretability and Generalizability

System-Wide Pollution of Biomedical Data: Consequence of the Search for Hub Genes of Hepatocellular Carcinoma Without Spatiotemporal Consideration

Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges

Contact Info

Product

Resources

About