Document images containing different types of information are required to be encrypted with different levels of security. In this paper, the image classification is carried out based on the feature extraction, for color images. The K-Nearest Neighbor (K-NN) method of image classification technique is used for classifying the query Document with trained set of features obtained from the Document database. Optical Character Recognition (OCR) technique is used to check for the presence as well as location of text/numerals in the Documents and to identify the Document type. Priority level is assigned in accordance with the Document type. Document images with different priorities are encrypted with different multi-dimensional chaotic maps. The Documents with different priority levels are diffused with different techniques. Document with highest priority are encrypted with highest level of security but Documents with lower priority levels are encrypted with lesser security levels. The proposed work was experimented for different document types with more number of image features for a large trained database. The results reveals a high speed of encryption for a set of document pages with priorities is more effective in comparison with a uniform method of encryption for all document types. The National Institute of Standards and Technology (NIST) statistical tests are also conducted to check for the randomness of the sequence and achieved good randomness. The proposed work also ensures security against the various statistical and differential attacks.