In recent years, the exponential growth of high dimensional, multi-modal molecular data has created both opportunities and challenges in personalized medicine. While existing approaches like matrix decomposition and neural network-based embeddings have been used to analyze such data, they have limitations in interpretability, handling missing values, and treating features across modalities as unrelated. To address these challenges, we present MUSIC (MUltiview BayeSIan Tensor DeComposition), a novel framework for probabilistic multi-view tensor decomposition that can integrate collections of tensors of different orders. MUSIC combines the strengths of group factor analysis and tensor decomposition through a Bayesian approach with structured sparsity priors. The framework offers several key advantages: (1) fast model training using variational inference, (2) inference of interpretable embeddings via structured sparsity, (3) efficient handling of missing values, and (4) flexible combination of tensors of different orders. We demonstrate MUSIC's effectiveness on both simulated data and real world applications, including drug response analysis in CLL patients and multi-modal single-cell data analysis in leukemia patients. Our results show that MUSIC can reveal interpretable multi-modal patterns capturing structured variation across patients, cell types, and modalities that are associated with disease states and can be explained through cell type- and modality-specific pathway activities.