Textual response generation is an essential task for multimodal task-oriented dialog systems. Although existing studies have achieved fruitful progress, they still suffer from two critical limitations: 1) focusing on the attribute knowledge but ignoring the relation knowledge that can reveal the correlations between different entities and hence promote the response generation, and 2) only conducting the cross-entropy loss based output-level supervision but lacking the representation-level regularization. To address these limitations, we devise a novel multimodal task-oriented dialog system (named MDS-S 2 ). Specifically, MDS-S 2 first simultaneously acquires the context related attribute and relation knowledge from the knowledge base, whereby the non-intuitive relation knowledge is extracted by the 𝑛-hop graph walk. Thereafter, considering that the attribute knowledge and relation knowledge can benefit the responding to different levels of questions, we design a multi-level knowledge composition module in MDS-S 2 to obtain the latent composed response representation. Moreover, we devise a set of latent query variables to distill the semantic information from the composed response representation and the ground truth response representation, respectively, and thus conduct the representation-level semantic regularization. Extensive experiments on a public dataset have verified the superiority of our proposed MDS-S 2 . We have released the codes and parameters to facilitate the research community.

Recently, studies on non-Hermitian topologic physics have attracted considerable attention. The non-Hermitian skin effect (NHSE), as a remarkable phenomenon in the non-Hermitian lattice, has been demonstrated in coupled ring resonators and photonic mesh lattices. However, there is a scarcity of work on the realization of NHSEs in synthetic dimensions, owing to inaccessible anisotropic coupling. This limits the potential for exploring non-Hermitian topologic physics in on-chip integrated optical systems. In this work, we implement a non-Hermitian Su–Schrieffer–Heeger topologic insulator in the synthetic frequency dimension, and the NHSE and topologic edge state are manifested. Furthermore, we demonstrate that the exotic chiral Zener tunneling can also be realized. Our system provides a versatile platform to explore and exploit non-Hermitian topologic physics on a chip and can have impacts on flexible light manipulation in frequency domains.

No abstract

No abstract

No abstract