Median Binary-Connect Method and a Binary Convolutional Neural Network for Word Recognition

Abstract

We propose and study a new projection formula for training binary weight convolutional neural networks. The projection formula measures the error in approximating a full precision (32 bit) vector by a 1-bit vector in the 1 norm instead of the standard 2 norm. The 1 projector is in closed analytical form and involves a median computation instead of an arithmatic average in the 2 projector. Experiments on 10 keywords classification show that the 1 (median) BinaryConnect (BC) method outperforms the regular BC, regardless of cold or warm start. The binary network trained by median BC and a recent blending technique reaches test accuracy 92.4 %, which is 1.1% lower than the full-precision network accuracy 93.5 %. On Android phone app, the trained binary network doubles the speed of full-precision network in spoken keywords recognition.

A Channel-Pruned and Weight-Binarized Convolutional Neural Network for Keyword Spotting

“…The CNN computation can speed up a lot if the weights are in the binary vector form: float precision scalar times a sign vector (· · · , ±1, ±1, · · · ), see [3]. For the keyword CNN, such weight binarization alone doubles the speed of an Android app that runs on Samsung Galaxy J7 cellular phone [5] with standard tensorflow functions such as 'conv2d' and 'matmul'.…”