The spatial leave-pair-out cross-validation method for reliable AUC estimation of spatial classifiers

Airola, Antti; Pohjankukka, Jonne; Torppa, Johanna; Middleton, Maarit; Nykänen, Vesa; Heikkonen, Jukka; Pahikkala, Tapio

doi:10.1007/s10618-018-00607-x

Cited by 17 publications

(6 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, their performance could have been better when applied to totally new data from a different time or different location. This is in line with prior research indicating that CCV often leads to models prone to overoptimistic and biased prediction results [9,17,51]. The SCV method produced worse results in the testing dataset than in B dataset (same area, another time).…”

Section: Discussionsupporting

confidence: 91%

“…Machine learning models that are robust and efficient at making realistic predictions typically assume that the data used to train them is independent and identically distributed (i.i.d.) [7,[9][10][11][12]. However, when this assumption is violated, it can lead to overfitting the highly flexible methods to the training data and underestimating spatial prediction errors [13].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Spatial or Random Cross-Validation? The Effect of Resampling Methods in Predicting Groundwater Salinity with Machine Learning in Mediterranean Region

Tziachris¹,

Nikou

Aschonitis³

et al. 2023

Water

View full text Add to dashboard Cite

Machine learning (ML) algorithms are extensively used with outstanding prediction accuracy. However, in some cases, their overfitting capabilities, along with inadvertent biases, might produce overly optimistic results. Spatial data are a special kind of data that could introduce biases to ML due to their intrinsic spatial autocorrelation. To address this issue, a special resampling method has emerged called spatial cross-validation (SCV). The purpose of this study was to evaluate the performance of SCV compared with conventional random cross-validation (CCV) used in most ML studies. Multiple ML models were created with CCV and SCV to predict groundwater electrical conductivity (EC) with data (A) from Rhodope, Greece, in the summer of 2020; (B) from the same area but at a different time (summer 2019); and (C) from a new area (the Salento peninsula, Italy). The results showed that the SCV provides ML models with superior generalization capabilities and, hence, better prediction results in new unknown data. The SCV seems to be able to capture the spatial patterns in the data while also reducing the over-optimism bias that is often associated with CCV methods. Based on the results, SCV could be applied with ML in studies that use spatial data.

show abstract

Section: Discussionsupporting

confidence: 91%

Section: Introductionmentioning

confidence: 99%

Spatial or Random Cross-Validation? The Effect of Resampling Methods in Predicting Groundwater Salinity with Machine Learning in Mediterranean Region

Tziachris¹,

Nikou

Aschonitis³

et al. 2023

Water

View full text Add to dashboard Cite

show abstract

“…The AUC is a criterion for estimating the probability that a classifier (predictor), i.e. gene-set can predict better than randomly selected classifiers 26 .…”

Section: Methodsmentioning

confidence: 99%

Identification of potentially functional modules and diagnostic genes related to amyotrophic lateral sclerosis based on the WGCNA and LASSO algorithms

Daneshafrooz

Cham

Majidi

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Amyotrophic lateral sclerosis (ALS) is a genetically and phenotypically heterogeneous disease results in the loss of motor neurons. Mounting information points to involvement of other systems including cognitive impairment. However, neither the valid biomarker for diagnosis nor effective therapeutic intervention is available for ALS. The present study is aimed at identifying potentially genetic biomarker that improves the diagnosis and treatment of ALS patients based on the data of the Gene Expression Omnibus. We retrieved datasets and conducted a weighted gene co-expression network analysis (WGCNA) to identify ALS-related co-expression genes. Functional enrichment analysis was performed to determine the features and pathways of the main modules. We then constructed an ALS-related model using the least absolute shrinkage and selection operator (LASSO) regression analysis and verified the model by the receiver operating characteristic (ROC) curve. Besides we screened the non-preserved gene modules in FTD and ALS-mimic disorders to distinct ALS-related genes from disorders with overlapping genes and features. Altogether, 4198 common genes between datasets with the most variation were analyzed and 16 distinct modules were identified through WGCNA. Blue module had the most correlation with ALS and functionally enriched in pathways of neurodegeneration-multiple diseases’, ‘amyotrophic lateral sclerosis’, and ‘endocytosis’ KEGG terms. Further, some of other modules related to ALS were enriched in ‘autophagy’ and ‘amyotrophic lateral sclerosis’. The 30 top of hub genes were recruited to a LASSO regression model and 5 genes (BCLAF1, GNA13, ARL6IP5, ARGLU1, and YPEL5) were identified as potentially diagnostic ALS biomarkers with validating of the ROC curve and AUC value.

show abstract

“…The former involves analyzing patterns of spatial distribution of toponyms in the text. De Bruijn et al have suggested a method for extracting toponyms by comparing databases of toponyms and OpenStreetMap (Airola et al, 2019). Current methods that rely on specific geographic directories are unable to identify unofficial place names mentioned in unstructured text.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Quantum-Perfected Model of Artificial Intelligence in the Problem of Automatic Recognition and Fast Conversion of Unstructured Text Information Into Spatial

STAROVOYT,

ZAYCHENKO

2023

AIT

View full text Add to dashboard Cite

Background. Efficiently converting large amounts of unstructured text data into spatial information is crucial for managing water distribution systems. This allows for the conversion of extensive sets of text information, such as reports, orders, letters, and other documents, into point classes of spatial objects in geographic information systems. To tackle this challenge, a promising new approach involves combining hybrid quantum-classical neural networks with geo-information technologies. Methods. The study utilized quantum-enhanced hybrid neural networks in combination with GIS methods to identify named entities such as personal accounts and balance sheet objects of Kyivvodokanal by their addresses and geocoding. This information was then published on a geoportal using the ArcGIS Enterprise platform in real-time, which holds great promise for effective water management. The performance of the developed model was evaluated by accuracy indicators, recall parameters, and weighted harmonic average of accuracy and recall. Results. The obtained results indicate that the developed hybrid quantum-classical model of artificial intelligence can be successfully applied to transform large volumes of unstructured textual information into spatial information. The model was integrated into GIS using ArcGIS Enterprise. By combining the obtained point classes of spatial objects with already existing data, methods of spatial connections, an interactive map with an update interval of every five minutes was developed. Conclusions. Taking advantage of quantum computing and combining it with classical hardware and classical AI models, it became possible to achieve similar and even better performance in various tasks compared to state-of-the-art methods. Quantum natural language processing is a promising new field that has the potential to revolutionize the way one analyzes and understands human language.

show abstract

The spatial leave-pair-out cross-validation method for reliable AUC estimation of spatial classifiers

Cited by 17 publications

References 37 publications

Spatial or Random Cross-Validation? The Effect of Resampling Methods in Predicting Groundwater Salinity with Machine Learning in Mediterranean Region

Spatial or Random Cross-Validation? The Effect of Resampling Methods in Predicting Groundwater Salinity with Machine Learning in Mediterranean Region

Identification of potentially functional modules and diagnostic genes related to amyotrophic lateral sclerosis based on the WGCNA and LASSO algorithms

A Hybrid Quantum-Perfected Model of Artificial Intelligence in the Problem of Automatic Recognition and Fast Conversion of Unstructured Text Information Into Spatial

Contact Info

Product

Resources

About