Learning low-dimensional nonlinear structures from high-dimensional noisy data: An integral operator approach

Ding, Xiucai; Ma, Rong

doi:10.1214/23-aos2306

Ann. Statist.

2023

DOI: 10.1214/23-aos2306

|View full text |Cite

Learning low-dimensional nonlinear structures from high-dimensional noisy data: An integral operator approach

Xiucai Ding,

Rong Ma

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

ADM: adaptive graph diffusion for meta-dimension reduction

Feng,

Liang,

2024

Briefings in Bioinformatics

View full text Add to dashboard Cite

Dimension reduction is essential for analyzing high-dimensional data, with various techniques developed to address diverse data characteristics. However, individual methods often struggle to capture all intricate patterns and complex structures simultaneously. To overcome this limitation, we introduce ADM (Adaptive graph Diffusion for Meta-dimension reduction), a novel meta-dimension reduction method grounded in graph diffusion theory. ADM integrates results from multiple dimension reduction techniques, leveraging their individual strengths while mitigating their specific weaknesses.ADM utilizes dynamic Markov processes to transform Euclidean space results into an information space, revealing intrinsic nonlinear manifold structures that are hard to capture by conventional methods. A critical advancement in ADM is its adaptive diffusion mechanism, which dynamically selects optimal diffusion time scales for each sample, enabling effective representation of multi-scale structures. This approach generates robust, high-quality low-dimensional representations that capture both local and global data structures while reducing noise and technique-specific distortions. We demonstrate ADM’s efficacy on simulated and real-world datasets, including various omics data types. Results show that ADM provides clearer separation between biological groups and reveals more meaningful patterns compared to existing methods, advancing the analysis and visualization of complex biological data.

show abstract

ADM: adaptive graph diffusion for meta-dimension reduction

Feng,

Liang,

2024

Briefings in Bioinformatics

View full text Add to dashboard Cite

show abstract

How do kernel-based sensor fusion algorithms behave under high-dimensional noise?

Ding,

2024

Information and Inference: A Journal of the IMA

View full text Add to dashboard Cite

We study the behavior of two kernel based sensor fusion algorithms, nonparametric canonical correlation analysis (NCCA) and alternating diffusion (AD), under the nonnull setting that the clean datasets collected from two sensors are modeled by a common low-dimensional manifold embedded in a high-dimensional Euclidean space and the datasets are corrupted by high-dimensional noise. We establish the asymptotic limits and convergence rates for the eigenvalues of the associated kernel matrices assuming that the sample dimension and sample size are comparably large, where NCCA and AD are conducted using the Gaussian kernel. It turns out that both the asymptotic limits and convergence rates depend on the signal-to-noise ratio (SNR) of each sensor and selected bandwidths. On one hand, we show that if NCCA and AD are directly applied to the noisy point clouds without any sanity check, it may generate artificial information that misleads scientists’ interpretation. On the other hand, we prove that if the bandwidths are selected adequately, both NCCA and AD can be made robust to high-dimensional noise when the SNRs are relatively large.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Learning low-dimensional nonlinear structures from high-dimensional noisy data: An integral operator approach

Cited by 2 publications

References 55 publications

ADM: adaptive graph diffusion for meta-dimension reduction

ADM: adaptive graph diffusion for meta-dimension reduction

How do kernel-based sensor fusion algorithms behave under high-dimensional noise?

Contact Info

Product

Resources

About