Device to Device (D2D) communication has emerged as the most promising paradigm for optimizing spectral and energy efficiency, reliability as well as increasing network throughput in the emerging cellular generation. D2D resource optimization scheme has been thoroughly investigated in the literature to manage interference and enable its smooth integration into the future cellular network in recent years as an increasing number of papers are published each year. We believe systematic categorization of literature in the area will help readers to comprehend the strengths, weaknesses, and trends of the solution approach. However, a survey that discusses all aspects, requirements, and challenges of D2D resource optimization is largely missing. Hence, in this paper, we aim to develop a comprehensive survey that fills the gaps found in the previous literature. Based on the surveyed papers, it will figure out ideas that have been thoroughly explored and those that assume the potential for further research.