Developing Video-Grounded Dialogue Systems (VGDS), where a dialogue is conducted based on visual and audio aspects of a given video, is significantly more challenging than traditional image or text-grounded dialogue systems because (1) feature space of videos span across multiple picture frames, making it difficult to obtain semantic information; and (2) a dialogue agent must perceive and process information from different modalities (audio, video, caption, etc.) to obtain a comprehensive understanding. Most existing work is based on RNNs and sequence-to-sequence architectures, which are not very effective for capturing complex long-term dependencies (like in videos). To overcome this, we propose Multimodal Transformer Networks (MTN) to encode videos and incorporate information from different modalities. We also propose queryaware attention through an auto-encoder to extract query-aware features from non-text modalities. We develop a training procedure to simulate token-level decoding to improve the quality of generated responses during inference. We get state of the art performance on Dialogue System Technology Challenge 7 (DSTC7). Our model also generalizes to another multimodal visual-grounded dialogue task, and obtains promising performance. We implemented our models using PyTorch and the code is released at https://github. com/henryhungle/MTN.

No abstract

No abstract

Naive T helper cells differentiate into functionally distinct effector subsets that drive specialized immune responses. Recent studies indicate that some of the effector subsets have plasticity. Here, we used an EAE model and found that Th17 cells deficient in the transcription factor BCL11B upregulated the Th2-associated proteins GATA3 and IL-4 without decreasing RAR-related orphan receptor γ (RORγt), IL-17, and GM-CSF levels. Surprisingly, abnormal IL-4 production affected Th17 cell trafficking, diverting migration from the draining lymph nodes/CNS route to the mesenteric lymph nodes/gut route, which ameliorated EAE without overt colitis. T helper cell rerouting in EAE was dependent on IL-4, which enhanced retinoic acid (RA) production by dendritic cells, which further induced expression of gut-homing receptors CCR9 and α 4 β 7 on Bcl11b-deficient CD4 + T cells. Furthermore, IL-4 treatment or Th2 immunization of wild-type mice with EAE caused no alteration in Th17 cytokines or RORγt, but diverted T helper cell trafficking to the gut, which improved EAE outcome without overt colitis. Our data demonstrate that Th17 cells are permissive to Th2 gene expression without affecting Th17 gene expression. This Th17 plasticity has an impact on trafficking, which is a critical component of the immune response and may represent a possible avenue for treating multiple sclerosis.

We have demonstrated a multistep 2-dimensional paper network immunoassay based on controlled rehydration of patterned, dried reagents. Previous work has shown that signal enhancement improves the limit of detection in 2-dimensional paper network assays, but until now, reagents have only been included as wet or dried in separate conjugate pads placed at the upstream end of the assay device. Wet reagents are not ideal for point-of-care because they must be refrigerated and typically limit automation and require more user steps. Conjugate pads allow drying but do not offer any control of the reagent distribution upon rehydration and can be a source of error when pads do not contact the assay membrane uniformly. Furthermore, each reagent is dried on a separate pad, increasing the fabrication complexity when implementing multistep assays that require several different reagents. Conversely, our novel method allows for consistent, controlled rehydration from patterned reagent storage depots directly within the paper membrane. In this assay demonstration, four separate reagents were patterned in different regions of the assay device: a gold-antibody conjugate used for antigen detection and three different signal enhancement components that must not be mixed until immediately before use. To show the viability of patterning and drying reagents directly onto a paper device for dry reagent storage and subsequent controlled release, we tested this device with the malaria antigen Plasmodium falciparum histidine-rich protein 2 (PfHRP2) as an example of target analyte. In this demonstration, the signal enhancement step increases the visible signal by roughly 3-fold and decreases the analytical limit of detection by 2.75-fold.