In order to increase the performance in the handwritten digit recognition field, researchers commonly combine a variety of features to represent a pattern. This approach has showed to be very effective in practice. The classical approach to combine features is by concatenating the underlying feature vectors. A drawback of this approach is that it could generate high-dimensional descriptors, which increases the complexity of the training process. Instead, we propose to use a pooling based classifier, that allow us to get not only a faster training process but also outperforming results. For evaluation, we used two state-of-the-art handwritten digit datasets: CVL and MNIST. In addition, we show that a simple rectangular spatial division, that characterize our descriptors, yields competitive results and a smaller computation cost with respect to other more complex zoning techniques.