Generative models have fascinated attention of researchers as it learns the important features from the trained data to generate the structures similar to the data provided for training. Autoencoders are the basic building components of generative models. In this work, we have designed a variational autoencoder to generate a large number of data to support the generative adversarial network model. The encoder, as well as the decoder of the proposed model, is designed using convolutional layers. The proposed method is verified on IIT Bhubaneswar Odia handwritten database. The encoder generates the feature vectors with the probability distribution of each category in latent space that follows the Gaussian distribution. It is verified that the decoder recognizes the features due to proper training. The generated images are quite similar to original data that validate the proposed VAE is well-generative. To measure the performance of the model, loss is calculated using binary cross-entropy along with Kullback-Leibler divergence loss. The proposed model is trained with Adam optimizer.