Various ocular diseases, such as cataract, diabetic retinopathy, and glaucoma have affected a large proportion of the population worldwide. In ophthalmology, fundus photography is used for the diagnosis of such retinal disorders. Nowadays, the setup of fundus image acquisition has changed from a fixed position to portable devices, making acquisition more vulnerable to distortions. However, a trustworthy diagnosis solely relies upon the quality of the fundus image. In recent years, fundus image quality assessment (IQA) has drawn much attention from researchers. This paper presents a detailed survey of the fundus IQA research. The survey covers a comprehensive discussion on the factors affecting the fundus image quality and the real-time distortions. The fundus IQA algorithms have been analyzed on the basis of the methodologies used and divided into three classes, namely: (i) Similarity-based, (ii) Segmentation-based, and (iii) Machine learning based. In addition, limitations of state of the art in this research field are also presented with the possible solutions. The objective of this paper is to provide a detailed information about the fundus IQA research with its significance, present status, limitations, and future scope. To the best of our knowledge, this is the first survey paper on the fundus IQA research.