With the increasing computational demands of neural networks, many hardware accelerators for the neural networks have been proposed. Such existing neural network accelerators often focus on popular neural network types such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs); however, not much attention has been paid to attention mechanisms, an emerging neural network primitive that enables neural networks to retrieve most relevant information from a knowledge-base, external memory, or past states. The attention mechanism is widely adopted by many state-of-the-art neural networks for computer vision, natural language processing, and machine translation, and accounts for a large portion of total execution time. We observe today's practice of implementing this mechanism using matrix-vector multiplication is suboptimal as the attention mechanism is semantically a content-based search where a large portion of computations ends up not being used. Based on this observation, we design and architect A 3 , which accelerates attention mechanisms in neural networks with algorithmic approximation and hardware specialization. Our proposed accelerator achieves multiple orders of magnitude improvement in energy efficiency (performance/watt) as well as substantial speedup over the state-of-the-art conventional hardware.

We investigate the existence and properties of Lipschitz solutions for some forward-backward parabolic equations in all dimensions. Our main approach to existence is motivated by reformulating such equations into partial differential inclusions and relies on a Baire's category method. In this way, the existence of infinitely many Lipschitz solutions to certain initial-boundary value problem of those equations is guaranteed under a pivotal density condition. Finally, we study two important cases of forward-backward anisotropic diffusion in which the density condition can be realized and therefore the existence results follow together with micro-oscillatory behavior of solutions. The first case is a generalization of the Perona-Malik model in image processing and the other that of Höllig's model related to the Clausius-Duhem inequality in the second law of thermodynamics.2010 Mathematics Subject Classification. 35M13, 35K20, 35D30, 49K20. Key words and phrases. forward-backward parabolic equations, partial differential inclusions, convex integration, Baire's category method, infinitely many Lipschitz solutions. 1 arXiv:1506.05847v1 [math.AP] 18 Jun 2015 Ω u(x, t)dx = Ω u 0 (x)dx ∀t ∈ [0, T ].

Abstract. We prove that for all smooth nonconstant initial data the initialNeumann boundary value problem for the Perona-Malik equation in image processing possesses infinitely many Lipschitz weak solutions on smooth bounded convex domains in all dimensions. Such existence results have not been known except for the one-dimensional problems. Our approach is motivated by reformulating the Perona-Malik equation as a nonhomogeneous partial differential inclusion with linear constraint and uncontrollable components of gradient. We establish a general existence result by a suitable Baire's category method under a pivotal density hypothesis. We finally fulfill this density hypothesis by convex integration based on certain approximations from an explicit formula of lamination convex hull of some matrix set involved.

The Perona-Malik equation is an ill-posed forward-backward parabolic equation with major application in image processing. In this paper we study the Perona-Malik type equation and show that, in all dimensions, there exist infinitely many radial weak solutions to the homogeneous Neumann boundary problem for any smooth nonconstant radially symmetric initial data. Our approach is to reformulate the n-dimensional equation into a one-dimensional equation, to convert the one-dimensional problem into a differential inclusion problem, and to apply a Baire's category method to generate infinitely many solutions.

Abstract. As a sequel to the paper [9], we study the existence and properties of Lipschitz solutions to the initial-boundary value problem of some forward-backward parabolic equations with diffusion fluxes violating Fourier's inequality.