Since the pioneering report with an unsupervised pre-training principle was published, deep architectures, as a simulation of primary cortexes, have been intensively studied and successfully utilized in solving some recognition tasks. Motivated by that, herein, we propose a decorrelating regularity on autoencoders, named decorrelating auto-encoder (DcA), which can be stacked to deep architectures, called the SDcA model. The learning algorithm is designed based on the principles of redundancy-reduction and the infomax, and a fine-tuning algorithm based on correlation detecting criteria. The property of our model is evaluated by auditory and handwriting recognition tasks with the TIMIT acoustic-phonetic continuous speech corpus and MNIST database. The results show that our model has a general advantage as compared with four existing models, especially in low levels, and when training samples are scarce our model put up stronger learning capacity and generalization. INDEX TERMS Machine learning, pattern recognition, deep architecture, auto-encoder, de-correlation.