We present an approach named JSFusion (Joint Sequence Fusion) that can measure semantic similarity between any pairs of multimodal sequence data (e.g. a video clip and a language sentence). Our multimodal matching network consists of two key components. First, the Joint Semantic Tensor composes a dense pairwise representation of two sequence data into a 3D tensor. Then, the Convolutional Hierarchical Decoder computes their similarity score by discovering hidden hierarchical matches between the two sequence modalities. Both modules leverage hierarchical attention mechanisms that learn to promote well-matched representation patterns while prune out misaligned ones in a bottom-up manner. Although the JSFusion is a universal model to be applicable to any multimodal sequence data, this work focuses on video-language tasks including multimodal retrieval and video QA. We evaluate the JS-Fusion model in three retrieval and VQA tasks in LSMDC, for which our model achieves the best performance reported so far. We also perform multiple-choice and movie retrieval tasks for the MSR-VTT dataset, on which our approach outperforms many state-of-the-art methods.

No abstract

Tuberculosis (TB) caused by Mycobacterium tuberculosis (Mtb) is one of the most deadly infectious diseases, with approximately two million people dying of TB annually. An effective therapeutic method for activating dendritic cells (DCs) and driving Th1 immune responses would improve host defenses and further the development of a TB vaccine. Given the importance of DC maturation in eliciting protective immunity against TB, we investigated whether Rv0315, a newly identified Mtb antigen, can prompt DC maturation. We found that Rv0315 functionally activated DCs by augmenting the expression of the co-stimulatory molecules CD80 and CD86 as well as MHC class I/II molecules. Moreover, it increased DC secretion of the pro-inflammatory cytokines IL-6, IL-1β, and TNF-α. Unlike LPS, however, Rv0315 induced the secretion of IL-12p70, but not IL-10. In addition, Rv0315-treated DCs accelerated the proliferation of CD4(+) and CD8(+) splenic T cells from Mtb-infected mice, with increased levels of IFN-γ, in syngeneic and allogeneic mixed lymphocyte reactions, indicating that Rv0315 contributes to Th1 polarization of the immune response. Importantly, both mitogen-activated protein kinases and nuclear factor κB signaling mediated the expression of DC surface markers and cytokines. Taken together, our results indicate that Rv0315 is a novel DC maturation-inducing antigen that drives T cell immune responses toward Th1 polarization, suggesting that Rv0315 plays a key role in determining the nature of the immune response to TB.

No abstract