Random Projections for Non-linear Dimensionality Reduction

Abstract

Abstract-The need to analyze high-dimensional data in various areas, such as image processing, human gene regulation and smart grids, raises the importance of dimensionality reduction. While classical linear dimensionality reduction methods are easily implementable and efficiently computable, they fail to discover the true structure of high-dimensional data lying on a non-linear subspace. To overcome this issue, many non-linear dimensionality reduction approaches, such as Locally Linear Embedding, Isometric Embedding and Semidefinite Embedding, have been proposed. Though these approaches can learn the global structure of non-linear manifolds, they are computationally expensive, potentially limiting their use in large-scale applications involve very high-dimensional data. An innovative method framework that combines random projections and non-linear dimensionality reduction methods is proposed in this paper to increase computational speed and reduce memory usage, while preserving the non-linear data geometry. Illustrations with various combinations of random projections and non-linear dimensionality reduction methods tested on a hand-written digits dataset are also given in this paper to demonstrate that this method framework is both computationally efficient and globally optimal.Index Terms-Random projections, nonlinear dimensionality reduction, manifold learning, high dimensional data.

End-to-end Spoken Conversational Question Answering: Task, Dataset and Model

“…In other words, we can distill the knowledge from one model (massive or teacher model) to another (small or student model). Previous work has shown that KD can significantly boost prediction accuracy in natural language processing and speech processing (Kim and Rush, 2016;Hu et al, 2018;Huang et al, 2018b;Hahn and Choi, 2019;Liu et al, 2021b,a;Cheng et al, 2016b;Cheng and You, 2016;Cheng et al, 2016a;You et al, 2020bYou et al, , 2021e, 2022bYou et al, , 2019aLyu et al, 2018Lyu et al, , 2019Guha et al, 2020;Yang et al, 2020;Ma et al, 2021a,b), while adopting KD-based methods for SQA tasks has been less explored. In this work, our goal is to handle the SCQA tasks.…”

Tuning Database-Friendly Random Projection Matrices for Improved Distance Preservation on Specific Data

“…-Non-linear Random Projections: RP-based techniques have been used to capture non-linear features in a compact representation. Approaches range from RP-based preprocessing for existing non-linear dimensionality reduction methods [22] to ad-hoc variants for non-linear kernel functions [5,23]. -Structured Johnson-Lindestrauss: Following the work of [24], structured JL methods try to approximate the result of a traditional RP by decomposing the projection matrix into a set of low-memory matrices [25,26].…”