Deep Leaning of the Neural Networks has become one of the most demanded areas of Information Technology and it has been successfully applied to solving many issues of Artificial Intelligence, for example, speech recognition, computer vision, natural language processing, data visualization. This paper describes the developing the deep neural network model for image recognition and a corresponding experimental research on an example of the MNIST data set. Some practical details for creating the Deep Neural Network and image recognition in the Caffe Framework are given as well.