With the commencement of new technologies like IoT and the Cloud, the sources of data generation have increased exponentially. The use and processing of this generated data have motivated and given birth to many other domains. The concept of a smart city has also evolved from making use of this data in decision-making in the various aspects of daily life and also improvement in the traditional systems. In smart cities, various technologies work collaboratively; they include devices used for data collection, processing, storing, retrieval, analysis, and decision making. Big data storage, retrieval, and analysis play a vital role in smart city applications. Traditional data processing approaches face many challenges when dealing with such voluminous and high-speed generated data, such as semi-structured or unstructured data, data privacy, security, real-time responses, and so on. Probabilistic Data Structures (PDS) has been evolved as a potential solution for many applications in smart cities to complete this tedious task of handling big data with real-time response. PDS has been used in many smart city domains, including healthcare, transportation, the environment, energy, and industry. The goal of this paper is to provide a comprehensive review of PDS and its applications in the domains of smart cities. The prominent domain of the smart city has been explored in detail; origin, current research status, challenges, and existing application of PDS along with research gaps and future directions. The foremost aim of this paper is to provide a detailed survey of PDS in smart cities; for readers and researchers who want to explore this field; along with the research opportunities in the domains.