Latent Space Geometric Statistics

Kühnel, Line; Fletcher, Tom; Joshi, Sarang; Sommer, Stefan

doi:10.1007/978-3-030-68780-9_16

Cited by 4 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For prediction training with the geodesic loss function, we set the number of interpolated points to be four. In addition, to speed up the algorithm for faster training, we ran the loop in Algorithm 1 for a fixed number of iterations (15)(16)(17)(18)(19)(20) instead of until convergence.…”

Section: Prediction Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Disentangling Multiple Features in Video Sequences using Gaussian Processes in Variational Autoencoders

Bhagat¹,

Uppal²,

Lim³

2020

Preprint

View full text Add to dashboard Cite

We introduce MGP-VAE, a variational autoencoder which uses Gaussian processes (GP) to model the latent space distribution. We employ MGP-VAE for the unsupervised learning of video sequences to obtain disentangled representations. Previous work in this area has mainly been confined to separating dynamic information from static content. We improve on previous results by establishing a framework by which multiple features, static or dynamic, can be disentangled. Specifically we use fractional Brownian motions (fBM) and Brownian bridges (BB) to enforce an inter-frame correlation structure in each independent channel. We show that varying this correlation structure enables one to capture different aspects of variation in the data. We demonstrate the quality of our disentangled representations on numerous experiments on three publicly available datasets, and also perform quantitative tests on a video prediction task. In addition, we introduce a novel geodesic loss function which takes into account the curvature of the data manifold to improve learning in the prediction task. Our experiments show quantitatively that the combination of our improved disentangled representations with the novel loss function enable MGP-VAE to outperform the state-of-theart in video prediction.

show abstract

Section: Prediction Resultsmentioning

confidence: 99%

“…Recent work has shown that that distances in latent space are not representative of the true distance between datapoints [1,19,25]. Rather, deep generative models learn a mapping from the latent space to the data manifold, a smoothly varying lower-dimensional subset of the original data space.…”

Section: Data Manifold Learningmentioning

confidence: 99%

Disentangling Multiple Features in Video Sequences using Gaussian Processes in Variational Autoencoders

Bhagat¹,

Uppal²,

Lim³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Given the high-dimensionality of the admission-level embeddings, we explore employing the dimensionality reduction algorithm UMAP to generate human-readable 2-dimensional plots of corpus manifolds for MIMIC-IV. The UMAP algorithm is known for preserving local structure between points while also maintaining more global structure compared to other dimensionality reduction methods like PCA 45 and t-SNE 46 . This property renders UMAP a valuable technique for interpreting encodings or embeddings of high-dimensional data and has been successfully applied in previous work to visualize biological datasets such as transcriptomes of single-cell data 47 .…”

Section: Methodsmentioning

confidence: 99%

Towards Maps of Disease Progression: Biomedical Large Language Model Latent Spaces For Representing Disease Phenotypes And Pseudotime

Zamora-Resendiz,

Khurram,

Crivelli

2024

Preprint

View full text Add to dashboard Cite

In this study, we propose a scientific framework to detect capability among biomedical large language models (LLMs) for organizing expressions of comorbid disease and temporal progression. We hypothesize that biomedical LLMs pretrained on next-token prediction produce latent spaces that implicitly capture "disease states" and disease progression, i.e., the transitions over disease states over time. We describe how foundation models may capture and transfer knowledge from explicit pretraining tasks to specific clinical applications. A scoring function based on Kullback-Leibler divergence was developed to measure "surprise" in seeing specialization when subsetting admissions along 13 biomedical LLM latent spaces. By detecting implicit ordering of longitudinal data, we aim to understand how these models self-organize clinical information and support tasks such as phenotypic classification and mortality prediction. We test our hypothesis along a case study for obstructive sleep apnea (OSA) in the publicly available MIMIC-IV dataset, finding ordering of phenotypic clusters and temporality within latent spaces. Our quantitative findings suggest that increased compute, conformance with compute-optimal training, and widening contexts promote better implicit ordering of clinical admissions by disease states, explaining 60.3% of the variance in our proposed implicit task. Preliminary qualitative findings suggest LLMs’ latent spaces trace patient trajectories through different phenotypic clusters, terminating at end-of-life phenotypes. This approach highlights the potential of biomedical LLMs in modeling disease progression, identifying new patterns in disease pathways and interventions, and evaluating clinical hypotheses related to drivers of severe illness. We underscore the need for larger, high-resolution longitudinal datasets to further validate and enhance understanding of the utility of LLMs in modeling patient trajectories along clinical text and advancing precision medicine.Key PointsQuestionDo LLMs sensibly organize cilnical data with respect to applications in precision medicine?FindingsBiomedically-trained LLMs show increasing potential in promoting the organization of patient data to reflect disease progression. In a subcohort of OSA patients, maps derived from LLMs’ latent representations reveal traceable disease trajectories.MeaningMaps of disease progression offer an explanation to the utility of LLMs in precision medicine. Following current pretraining conventions in foundation modeling, scientific inquiry into these maps may help anticipate progress in applications of LLMs for healthcare.

show abstract

“…The generator's core mandate is two-fold: firstly, it is tasked with the application of a feature extraction module to effectuate the mapping of input samples into latent space. In this process, vital feature information is meticulously encoded into latent vectors [42][43][44]. Subsequently, the generator employs an image reconstruction module to undertake the vital task of decoding feature information from these latent vectors, culminating in the meticulous reconstruction of the dehazed image.…”

Section: Methodsmentioning

confidence: 99%

Foggy image restoration using deep sub‐pixel reconstruction network

Li,

Liu,

Shi

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

Light undergoes attenuation due to scattering and refraction when propagating through aerosols. In foggy conditions, Aerosol particles in the troposphere exhibit high mobility introducing intricate non‐linear noise into images. Foggy image restoration represents an ill‐posed problem, where traditional physical models and image enhancement techniques often prove inadequate in delivering effective solutions. This paper introduces a novel deep sub‐pixel reconstruction algorithm for foggy image restoration, pioneering the application of sub‐pixel reconstruction modules to this domain. This model employs convolutional layers to extract low‐level features and dense‐connected layers for high‐level feature extraction. Furthermore, a specialized sub‐pixel reconstruction module tailored for the task of foggy image restoration is designed, with the purpose of reconstructing dehazed images from latent vectors. During training, a generative adversarial training framework is adopted, incorporating a purpose‐designed discriminator. Additionally, a fusion loss is implemented to facilitate model refinement. Quantitative and qualitative evaluation experiments conducted on synthetic and real‐world image datasets demonstrate the effectiveness of the proposed method in preserving finer details. The Structural Similarity Index (SSIM) is observed to improve by 2.5%, attesting to enhanced perceptual quality for grayscale foggy images.

show abstract

Latent Space Geometric Statistics

Cited by 4 publications

References 17 publications

Disentangling Multiple Features in Video Sequences using Gaussian Processes in Variational Autoencoders

Disentangling Multiple Features in Video Sequences using Gaussian Processes in Variational Autoencoders

Towards Maps of Disease Progression: Biomedical Large Language Model Latent Spaces For Representing Disease Phenotypes And Pseudotime

Foggy image restoration using deep sub‐pixel reconstruction network

Contact Info

Product

Resources

About