In recent years, many systems have been developed to embed deep learning in robots. Some use multimodal information to achieve higher accuracy. In this paper, we highlight three aspects of such systems: cost, robustness, and system optimization. First, because the optimization of large architectures using real environments is computationally expensive, developing such architectures is difficult. Second, in a real-world environment, noise, such as changes in lighting, is often contained in the input. Thus, the architecture should be robust against noise. Finally, it can be difficult to coordinate a system composed of individually optimized modules; thus, the system is better optimized as one architecture. To address these aspects, a simple and highly robust architecture, namely memorizing and associating converted multimodal signal architecture (MACMSA), is proposed in this study. Verification experiments are conducted, and the potential of the proposed architecture is discussed. The experimental results show that MACMSA diminishes the effects of noise and obtains substantially higher robustness than a simple autoencoder. MACMSA takes us one step closer to building robots that can truly interact with humans.

In recent years, models that integrate multimodal information to control robots have been actively developed. Memorizing and Associating Converted Multimodal Signal Architecture (MACMSA) was proposed to integrate multimodal information obtained from robots with Hopfield networks as associators and independent feed-forward neural networks as encoders and decoders. The performance of MACMSA has thus far been investigated only using pseudo-data. Notably, MACMSA exhibits high resistance to noise. However, it cannot generate signals for robot control. The purpose of this study was to improve MACMSA to generate signals for robot control and optimize it using real data on reaching tasks. The results of the generated control signals on a real machine are presented to demonstrate that the improved model can be effectively used in a real environment. The results also show that the proposed model can perform well with real data.