Offline handwritten character recognition (OHCR) is considered a challenging task in pattern recognition due to the inter-class similarity and intra-class variations among the symbols present in the alphabet set. In this work, a learningbased weighted average ensemble of deep neural network models (WEnDNN) is proposed to classify the 10 digits and 47 characters present in the alphabet set of Odia language, an official language of India. To build the base model for the ensemble network (EnDNN), three suitable convolutional neural networks (CNN), are designed and trained from scratch. The WEnDNN's accuracy is increased by using a grid search approach to determine the ideal weight allocations to give to the top-performing model. The performance of the WEnDNN model is compared with several standard machine learning models, which take the nonhandcrafted features extracted from the finely tuned, pre-trained VGG16 model and a combination of Gabor and pixel intensity values to create handcrafted features. On several benchmark handwritten datasets, including NITR Odia characters (OHCS v1.0), ISI Kolkata Odia numerals, and IITBBS Odia numerals, the performance of the proposed WEnDNN model is assessed and compared. The experimental results demonstrate that, in terms of recognition accuracy, the proposed approach beats other state-of-the-art approaches.