In this paper, we are interested in modeling complex activities that occur in a typical household. We propose to use programs, i.e., sequences of atomic actions and interactions, as a high level representation of complex tasks. Programs are interesting because they provide a non-ambiguous representation of a task, and allow agents to execute them. However, nowadays, there is no database providing this type of information. Towards this goal, we first crowd-source programs for a variety of activities that happen in people's homes, via a game-like interface used for teaching kids how to code. Using the collected dataset, we show how we can learn to extract programs directly from natural language descriptions or from videos. We then implement the most common atomic (inter)actions in the Unity3D game engine, and use our programs to "drive" an artificial agent to execute tasks in a simulated household environment. Our VirtualHome simulator allows us to create a large activity video dataset with rich ground-truth, enabling training and testing of video understanding models. We further showcase examples of our agent performing tasks in our VirtualHome based on language descriptions.

We address the problem of affordance reasoning in diverse scenes that appear in the real world. Affordances relate the agent's actions to their effects when taken on the surrounding objects. In our work, we take the egocentric view of the scene, and aim to reason about action-object affordances that respect both the physical world as well as the social norms imposed by the society. We also aim to teach artificial agents why some actions should not be taken in certain situations, and what would likely happen if these actions would be taken. We collect a new dataset that builds upon ADE20k [32], referred to as ADE-Affordance, which contains annotations enabling such rich visual reasoning. We propose a model that exploits Graph Neural Networks to propagate contextual information from the scene in order to perform detailed affordance reasoning about each object. Our model is showcased through various ablation studies, pointing to successes and challenges in this complex task.

No abstract

No abstract

LysR-type transcriptional regulators (LTTRs) are ubiquitous and abundant amongst bacteria and control a variety of cellular processes. Here, we investigated the effect of Rsc1880 (a putative LTTR, hereafter designated as PrhO) on the pathogenicity of Ralstonia solanacearum. Deletion of prhO substantially reduced the expression of the type III secretion system (T3SS) both in vitro and in planta, and resulted in significantly impaired virulence in tomato and tobacco plants. Complementary prhO completely restored the reduced virulence and T3SS expression to that of the wild-type. Moreover, PrhO-dependent T3SS and virulence were conserved amongst R. solanacearum species. However, deletion of prhO did not alter biofilm formation, swimming mobility and in planta growth. The expression of some type III effectors was significantly reduced in prhO mutants, but the hypersensitive response was not affected in tobacco leaves. Consistent with the key regulatory role of HrpB on T3SS, PrhO positively regulated the T3SS through HrpB. Furthermore, PrhO regulated hrpB expression via two close paralogues, HrpG and PrhG, which are two-component response regulators and positively regulate hrpB expression in a parallel manner. However, deletion of prhO did not alter the expression of phcA, prhJ and prhN, which are also involved in hrpB regulation. In addition, PrhO was expressed in a cell density-dependent manner, but negatively repressed by itself. No regulation was observed for HrpB, PhcA and PrhN on prhO expression. Taken together, we genetically demonstrated that PrhO is a novel virulence regulator of R. solanacearum, which positively regulates T3SS expression through HrpG, PrhG and HrpB and contributes to virulence.