No abstract

No abstract

No abstract

Machine comprehension of visual information from images and videos by neural networks suffers from two limitations: (1) the computational and inference gap in vision and language to accurately determine which object a given agent acts on and then to represent it by language, and (2) the shortcoming in stability and generalization of the classifier trained by a single, monolithic neural network. To address these limitations, we propose MoE-VRD, a novel approach to visual relationship detection via a mixture of experts. MoE-VRD recognizes language triplets in the form of a < subject, predicate, object > tuple to extract the relationship between subject, predicate, and object from visual processing. Since detecting a relationship between a subject (acting) and the object(s) (being acted upon) requires that the action be recognized, we base our network on recent work in visual relationship detection. To address the limitations associated with single monolithic networks, our mixture of experts is based on multiple small models, whose outputs are aggregated. That is, each expert in MoE-VRD is a visual relationship learner capable of detecting and tagging objects. MoE-VRD employs an ensemble of networks while preserving the complexity and computational cost of the original underlying visual relationship model by applying a sparsely-gated mixture of experts, which allows for conditional computation and a significant gain in neural network capacity. We show that the conditional computation capabilities and massive ability to scale the mixture-of-experts leads to an approach to the visual relationship detection problem which outperforms the state-of-the-art.

In order to maintain productivity and alertness, individuals must be thermally comfortable in the space they occupy (whether it is a cubicle, a room, a car, etc.). However, it is often difficult to non-intrusively assess an occupant's "thermal comfort" and hence most heating, ventilation, and air conditioning (HVAC) engineers adopt fixed temperature settings to "err on the safe side". These set temperatures can be too hot or too cold for individuals wearing different clothing, and as a result lead to feelings of discomfort as well as wastage of energy. To address these challenges, we develop SiCILIA, a platform that extracts physical and personal variables of an occupant's thermal environment to infer the amount of clothing insulation without human intervention. Clothing insulation is one of the most influential factors in determining thermal comfort. The proposed inference algorithm builds upon theories of body heat transfer, and is corroborated by empirical data. Experimental results show that the algorithm is capable of accurately predicting an occupant's thermal insulation with a mean prediction error of approximately 0.2 clo.