Multimodal factor analysis

Yılmaz, Yasin; Hero, Alfred O.

doi:10.1109/mlsp.2015.7324340

Cited by 3 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4 depicts the proposed generative latent variable model. The proposed model can be regarded as a multimodal factor analysis model [19] since it combines features from two disparate domains(geotag and text). In classical factor analysis [31], the mean of a Gaussian random variable is modeled with the linear combination c T u of factor scores in u, where the coefficients in c are called factor loadings.The number of factors is typically much less than the number of variables modeled, as K ≪ P in our case.…”

Section: Generative Latent Variable Modelmentioning

confidence: 99%

“…Recall that η i = Ũ c i = K k=1 c ik ũk . In (24), the latent variable vectors { ũk }, which are independently modeled a priori (19), are coupled, thus no more independent a posteriori in P { ũk }|{ hi }, θ . To capture the dependency we treat them in a single vector ũ = [ ũ(1) .…”

Section: E-stepmentioning

confidence: 99%

“…To fuse the multimodal data we use a probabilistic generative model, and derive an expectation-maximization (EM) algorithm to find the maximum likelihood (ML) estimates of the model parameters. The proposed model can be seen as a multimodal factor analysis model [19,20]. However, it is more general than the model in [20] in terms of the considered probabilistic models, and also the temporal dimension that is inherent to our problem.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multimodal Event Detection in Twitter Hashtag Networks

Yılmaz

Hero

2016

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

Event detection in a multimodal Twitter dataset is considered. We treat the hashtags in the dataset as instances with two modes: text and geolocation features. The text feature consists of a bag-of-words representation. The geolocation feature consists of geotags (i.e., geographical coordinates) of the tweets. Fusing the multimodal data we aim to detect, in terms of topic and geolocation, the interesting events and the associated hashtags. To this end, a generative latent variable model is assumed, and a generalized expectation-maximization (EM) algorithm is derived to learn the model parameters. The proposed method is computationally efficient, and lends itself to big datasets. Experimental results on a Twitter dataset from August 2014 show the efficacy of the proposed method.

show abstract

Section: Generative Latent Variable Modelmentioning

confidence: 99%

Section: E-stepmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Event Detection in Twitter Hashtag Networks

Yılmaz

Hero

2016

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

show abstract

“…This paper develops a general approach to joint factor analysis for heterogeneous data, called multimodal factor analysis (MMFA). MMFA, originally introduced in [26], is a Bayesian approach that models different types of data using latent factor loadings specific to each data type and latent factor scores that are common to the data types. It was applied to event detection in Twitter [27] in order to fuse categorical and spherical data that are modeled by multinomial and von Mises-Fisher distributions, respectively.…”

Section: Introductionmentioning

confidence: 99%

“…• As opposed to the preliminary work [26], [27], the proposed generalized MMFA model provides a tractable unified framework for jointly analyzing heterogeneous features from the exponential family of distributions through linking their natural parameters with a common factor score vector. • Motivated by the Bernstein-von Mises theorem [35] MMFA finds Gaussian approximations to the posterior distribution of latent factor loadings for the data types whose natural parameters require nonlinear link functions, e.g., multinomial distribution (a.k.a.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Data Fusion in High-Dimensional Heterogeneous Datasets via Generative Models

Yilmaz,

Aktukmak,

Hero

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The commonly used latent space embedding techniques, such as Principal Component Analysis, Factor Analysis, and manifold learning techniques, are typically used for learning effective representations of homogeneous data. However, they do not readily extend to heterogeneous data that are a combination of numerical and categorical variables, e.g., arising from linked GPS and text data. In this paper, we are interested in learning probabilistic generative models from high-dimensional heterogeneous data in an unsupervised fashion. The learned generative model provides latent unified representations that capture the factors common to the multiple dimensions of the data, and thus enable fusing multimodal data for various machine learning tasks. Following a Bayesian approach, we propose a general framework that combines disparate data types through the natural parameterization of the exponential family of distributions. To scale the model inference to millions of instances with thousands of features, we use the Laplace-Bernstein approximation for posterior computations involving nonlinear link functions. The proposed algorithm is presented in detail for the commonly encountered heterogeneous datasets with real-valued (Gaussian) and categorical (multinomial) features. Experiments on two high-dimensional and heterogeneous datasets (NYC Taxi and MovieLens-10M) demonstrate the scalability and competitive performance of the proposed algorithm on different machine learning tasks such as anomaly detection, data imputation, and recommender systems.

show abstract