Iktishaf+: A Big Data Tool with Automatic Labeling for Road Traffic Social Sensing and Event Detection Using Distributed Machine Learning

Abstract

Digital societies could be characterized by their increasing desire to express themselves and interact with others. This is being realized through digital platforms such as social media that have increasingly become convenient and inexpensive sensors compared to physical sensors in many sectors of smart societies. One such major sector is road transportation, which is the backbone of modern economies and costs globally 1.25 million deaths and 50 million human injuries annually. The cutting-edge on big data-enabled social media analytics for transportation-related studies is limited. This paper brings a range of technologies together to detect road traffic-related events using big data and distributed machine learning. The most specific contribution of this research is an automatic labelling method for machine learning-based traffic-related event detection from Twitter data in the Arabic language. The proposed method has been implemented in a software tool called Iktishaf+ (an Arabic word meaning discovery) that is able to detect traffic events automatically from tweets in the Arabic language using distributed machine learning over Apache Spark. The tool is built using nine components and a range of technologies including Apache Spark, Parquet, and MongoDB. Iktishaf+ uses a light stemmer for the Arabic language developed by us. We also use in this work a location extractor developed by us that allows us to extract and visualize spatio-temporal information about the detected events. The specific data used in this work comprises 33.5 million tweets collected from Saudi Arabia using the Twitter API. Using support vector machines, naïve Bayes, and logistic regression-based classifiers, we are able to detect and validate several real events in Saudi Arabia without prior knowledge, including a fire in Jeddah, rains in Makkah, and an accident in Riyadh. The findings show the effectiveness of Twitter media in detecting important events with no prior knowledge about them.

Green Artificial Intelligence: Towards an Efficient, Sustainable and Equitable Technology for Smart Cities and Futures

“…A physical sensor to measure the sustainability of urban infrastructure and environmental pollution can be considered a green sensor [92,93]. A virtual sensor, such as that using big data or social media data to detect congestion on the road, can be considered a green sensor for environmental sustainability, in the sense that it detects the environmental pollution that may be caused by a high intensity of pollution in the geographical area where congestion is happening [94].…”

Detecting Natural Hazard-Related Disaster Impacts with Social Media Analytics: The Case of Australian States and Territories

“…In recent years, big data social media analytics has become a popular approach to investigate various urban issues [45]. For instance, some studies focused on multiple human languages and a range of diverse topics including COVID-19-related governance matters [46,47], while others explored traffic-related event detection [48], detecting critical diseases and symptoms in and across cities [49]. Additionally, some scholars used big data social media analytics in examining urban logistics [50], public sentiment analyses of government services [51], and disaster severity prediction [23].…”

DNA Profiling: An Investigation of Six Machine Learning Algorithms for Estimating the Number of Contributors in DNA Mixtures

“…Moreover, we will use deep learning to improve classification performance. A major theme of our research is smart cities and societies [23]- [25], big data [26]- [28], high performance computing [29], [30], healthcare [31]- [33], information systems [34], [35], system integration [36], [37], and artificial intelligence [38], [39]. Future work on DNA profiling will also look into developing new smart applications for DNA profiling and its integration with other smart city systems.…”