An accurate diagnosis is significant for the treatment of any disease in its early stage. Content-Based Medical Image Retrieval (CBMIR) is used to find similar medical images in a huge database to help radiologists in diagnosis. The main difficulty in CBMIR is semantic gaps between the lower-level visual details, captured by computer-aided tools and higher-level semantic details captured by humans. Many existing methods such as Manhattan Distance, Triplet Deep Hashing, and Transfer Learning techniques for CBMIR were developed but showed lower efficiency and the computational cost was high. To solve such issues, a new feature extraction approach is proposed using Histogram of Gradient (HoG) with Local Ternary Pattern (LTP) to automatically retrieve medical images from the Contrast-Enhanced Magnetic Resonance Imaging (CE-MRI) database. Adam optimization algorithm is utilized to select features and the Euclidean measure calculates the similarity for query images. From the experimental analysis, it is clearly showing that the proposed HoG-LTP method achieves higher accuracy of 98.8%, a sensitivity of 98.5%, and a specificity of 99.416%, which is better when compared to the existing Random Forest (RF) method which displayed an accuracy, sensitivity, and specificity of 81.1%, 81.7% and 90.5% respectively.