Diabetes mellitus is a major medical concern worldwide. Long-term diabetes can affect the retina of the eye and lead to diabetic retinopathy (DR) and diabetic macular edema (DME). Proper screening and consultation with an ophthalmologist are necessary to prevent avoidable vision loss. As DR and DME have become more prevalent, automated screening is essential to provide costeffective and rapid solutions with reduced human resources requirements. This paper aims to provide a comprehensive review of the literature on computer-aided diagnosis of DR and DME. We identified the studies on automated five-class grading of DR according to International Clinical Diabetic Retinopathy severity scale and three class grading of diabetic maculopathy, using fundus images. A systematic search on research repositories was conducted, and relevant studies were scrutinized and included in the review. The studies were reported in nearly 100 different journals. We have reviewed the studies in all aspects including datasets, preprocessing, non-deep learning, and deep learning-based algorithms, and evaluation metrics. Significant contributions in developing automated tools for DR/DME grading are highlighted. We have identified and discussed research gaps and challenges. This will help researchers to get an updated summary of work done in the area. Deep learning-based algorithms have outperformed the traditional algorithms in the domain. Despite their promising performance, these algorithms reveal the potential for significant improvements to become a reliable tool in clinical settings.