Livestock is a critical socioeconomic asset in developing countries such as Ethiopia, where the economy is significantly based on agriculture and animal husbandry. However, there is an enormous loss of livestock population, which undermines efforts to achieve food security and poverty reduction in the country. The primary reason for this challenge is the lack of a reliable and prompt diagnosis system that identifies livestock diseases in a timely manner. To address some of these issues, the integration of an expert system with deep learning image processing was proposed in this study. Due to the economic significance of cattle in Ethiopia, this study was only focused on cattle disease diagnosis. The cattle disease symptoms that were visible to the naked eye were collected by a cell phone camera. Symptoms that were identified by palpation were collected by text dialogue. The identification of the symptoms category was performed by the image analysis component using a convolutional neural network (CNN) algorithm. The algorithm classified the input symptoms with 95% accuracy. The final diagnosis conclusion was drawn by the reasoner component of the expert system by integrating image classification results, location, and text information obtained from the users. We developed a prototype system that incorporates the image classification algorithms and the reasoner component. The evaluation result of the developed system showed that the new diagnosis system could provide a rapid and effective diagnosis of cattle diseases.