Machine learning is penetrating most of the classification and recognition tasks performed by a computer. This paper proposes the classification of flower images using a powerful artificial intelligence tool, convolutional neural networks (CNN). A flower image database with 9500 images is considered for the experimentation. The entire database is sub categorized into 4. The CNN training is initiated in five batches and the testing is carried out on all the for datasets. Different CNN architectures were designed and tested with our flower image data to obtain better accuracy in recognition. Various pooling schemes were implemented to improve the classification rates. We achieved 97.78% recognition rate compared to other classifier models reported on the same dataset.