Modeling Long-Term Dependencies from Videos Using Deep Multiplicative Neural Networks

Abstract

Understanding temporal dependencies of videos is fundamental for vision problems, but deep learning–based models are still insufficient in this field. In this article, we propose a novel deep multiplicative neural network (DMNN) for learning hierarchical long-term representations from video. The DMNN is built upon the multiplicative block that remembers the pairwise transformations between consecutive frames using multiplicative interactions rather than the regular weighted-sum ones. The block is slided over the timesteps to update the memory of the networks on the frame pairs. Deep architecture can be implemented by stacking multiple layers of the sliding blocks. The multiplicative interactions lead to exact, rather than approximate, modeling of temporal dependencies. The memory mechanism can remember the temporal dependencies for an arbitrary length of time. The multiple layers output multiple-level representations that reflect the multi-timescale structure of video. Moreover, to address the difficulty of training DMNNs, we derive a theoretically sound convergent method, which leads to a fast and stable convergence. We demonstrate a new state-of-the-art classification performance with proposed networks on the UCF101 dataset and the effectiveness of capturing complicate temporal dependencies on a variety of synthetic datasets.

Improving the Use of Blockchain Technology in Stroke Care Information Management Systems

“…It fosters trust in an untrustworthy competitive atmosphere while costing very little, and blockchain technology is seen as a breakthrough development in computing. As a result of the rapid advancement of new technology, industries all around the world have been reorganised [ 2 ]. The blockchain, which is increasingly being used by other firms, is built on encrypted digital currency.…”

Design and Application of Electronic Rehabilitation Medical Record (ERMR) Sharing Scheme Based on Blockchain Technology

“…A safe, reliable, and easily accessible ERMR will definitely improve the work efficiency of rehabilitation doctors, facilitate the rehabilitation doctors to accurately understand the patient's personalized information and rehabilitation needs, adopt targeted rehabilitation treatment plans, and improve the effectiveness of individual rehabilitation [ 1 ]. The sharing of ERMR is also beneficial to rehabilitation research institutions and provides basic data and case references for prospective scientific research [ 2 , 3 ].…”