Automatic detection of glaucoma from retinal fundus images is a major area of research in Computer aided diagnostics. In this paper, we propose a novel methodology to detect glaucoma by using energy of graph concepts. The retinal fundus image is first segmented to get the structure of the retinal vasculature using various image processing techniques. The retinal vasculature is then modeled into two graphs based on the position of branchpoints and the crossover points in the image. The graphs thus formed are simplified and 6 different energies are extracted from it. These energies are then used as features to a machine learning model i.e., quadratic support vector machine after performing principal component analysis. The methodology was tested out on the G1020 dataset and the results obtained show that energy of graphs does contain discriminating information regarding disease detection.