Real time road traffic event detection using Twitter and spark

Pandhare, Ketan R.; Shah, Medha A.

doi:10.1109/icicct.2017.7975237

Cited by 25 publications

(12 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The core ingredients of these digital societies, or digital platforms that enable these societies include a range of emerging technologies and their convergence. The technologies include big data [ 27 , 28 , 29 , 30 ], high-performance computing (HPC) [ 31 , 32 , 33 , 34 ], artificial intelligence [ 35 , 36 , 37 , 38 ], cloud, fog, and edge computing [ 39 , 40 , 41 ], social sensing [ 42 , 43 , 44 , 45 ], and Internet of Things (IoT) [ 36 , 40 , 46 , 47 , 48 ]. The applications include transportation [ 4 , 49 , 50 , 51 , 52 , 53 ], healthcare [ 7 , 54 , 55 , 56 ], and others [ 57 , 58 , 59 , 60 ].…”

Section: Literature Reviewmentioning

confidence: 99%

Iktishaf+: A Big Data Tool with Automatic Labeling for Road Traffic Social Sensing and Event Detection Using Distributed Machine Learning

Alomari

Katib

Albeshri

et al. 2021

Sensors

View full text Add to dashboard Cite

Digital societies could be characterized by their increasing desire to express themselves and interact with others. This is being realized through digital platforms such as social media that have increasingly become convenient and inexpensive sensors compared to physical sensors in many sectors of smart societies. One such major sector is road transportation, which is the backbone of modern economies and costs globally 1.25 million deaths and 50 million human injuries annually. The cutting-edge on big data-enabled social media analytics for transportation-related studies is limited. This paper brings a range of technologies together to detect road traffic-related events using big data and distributed machine learning. The most specific contribution of this research is an automatic labelling method for machine learning-based traffic-related event detection from Twitter data in the Arabic language. The proposed method has been implemented in a software tool called Iktishaf+ (an Arabic word meaning discovery) that is able to detect traffic events automatically from tweets in the Arabic language using distributed machine learning over Apache Spark. The tool is built using nine components and a range of technologies including Apache Spark, Parquet, and MongoDB. Iktishaf+ uses a light stemmer for the Arabic language developed by us. We also use in this work a location extractor developed by us that allows us to extract and visualize spatio-temporal information about the detected events. The specific data used in this work comprises 33.5 million tweets collected from Saudi Arabia using the Twitter API. Using support vector machines, naïve Bayes, and logistic regression-based classifiers, we are able to detect and validate several real events in Saudi Arabia without prior knowledge, including a fire in Jeddah, rains in Makkah, and an accident in Riyadh. The findings show the effectiveness of Twitter media in detecting important events with no prior knowledge about them.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Iktishaf+: A Big Data Tool with Automatic Labeling for Road Traffic Social Sensing and Event Detection Using Distributed Machine Learning

Alomari

Katib

Albeshri

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…A range of technologies is contributing to the development of these smart systems. These include the Internet of Things (IoT) [32][33][34][35][36], social media [21][22][23]37,38], big data [39][40][41][42][43][44], high performance computing (HPC) [45][46][47][48], cloud, fog, and edge computing [34,[49][50][51][52], and machine learning [36,53]. The applications include healthcare [34,39,[54][55][56], transportation [57,58], and others [59,60].…”

Section: Literature Reviewmentioning

confidence: 99%

“…The challenges in this respect include management, integration, and distributed computation of data including the difficulties related to managing the 4V characteristics of big data, i.e., volume, velocity, variety, and veracity. There have been some works on the use of big data platforms in Twitter data analytics in various languages but in different application domains [18][19][20][21][22][23]37,78,79]. To the best of our knowledge, no work has been reported that uses big data technologies for data analytics in healthcare using tweets in the Arabic language.…”

Section: Research Gapmentioning

confidence: 99%

Sehaa: A Big Data Analytics Tool for Healthcare Symptoms and Diseases Detection Using Twitter, Apache Spark, and Machine Learning

et al. 2020

View full text Add to dashboard Cite

Smartness, which underpins smart cities and societies, is defined by our ability to engage with our environments, analyze them, and make decisions, all in a timely manner. Healthcare is the prime candidate needing the transformative capability of this smartness. Social media could enable a ubiquitous and continuous engagement between healthcare stakeholders, leading to better public health. Current works are limited in their scope, functionality, and scalability. This paper proposes Sehaa, a big data analytics tool for healthcare in the Kingdom of Saudi Arabia (KSA) using Twitter data in Arabic. Sehaa uses Naive Bayes, Logistic Regression, and multiple feature extraction methods to detect various diseases in the KSA. Sehaa found that the top five diseases in Saudi Arabia in terms of the actual afflicted cases are dermal diseases, heart diseases, hypertension, cancer, and diabetes. Riyadh and Jeddah need to do more in creating awareness about the top diseases. Taif is the healthiest city in the KSA in terms of the detected diseases and awareness activities. Sehaa is developed over Apache Spark allowing true scalability. The dataset used comprises 18.9 million tweets collected from November 2018 to September 2019. The results are evaluated using well-known numerical criteria (Accuracy and F1-Score) and are validated against externally available statistics.

show abstract

“…Then the algorithm detects upcoming popular events as well as predicts the irregular behaviour of UEs. The paper [17] proposed a context-aware load balancing based on predicting an event in simulation, and [153] also studied that unexpected real-time road traffic prediction and control based the Tweets by waiting drivers.…”

Section: Predictive User Behaviour 1) Seasonality In User Behaviourmentioning

confidence: 99%

A Survey of Online Data-Driven Proactive 5G Network Optimisation Using Machine Learning

Guo

Zhang

2020

IEEE Access

View full text Add to dashboard Cite

In the fifth-generation (5G) mobile networks, proactive network optimisation plays an important role in meeting the exponential traffic growth, more stringent service requirements, and to reduce capital and operational expenditure. Proactive network optimisation is widely acknowledged as one of the most promising ways to transform the 5G network based on big data analysis and cloud-fog-edge computing, but there are many challenges. Proactive algorithms will require accurate forecasting of highly contextualised traffic demand and quantifying the uncertainty to drive decision making with performance guarantees. Context in Cyber-Physical-Social Systems (CPSS) is often challenging to uncover, unfolds over time, and even more difficult to quantify and integrate into decision making. The first part of the review focuses on mining and inferring CPSS context from heterogeneous data sources, such as online user-generated-content. It will examine the state-of-the-art methods currently employed to infer location, social behaviour, and traffic demand through a cloud-edge computing framework; combining them to form the input to proactive algorithms. The second part of the review focuses on exploiting and integrating the demand knowledge for a range of proactive optimisation techniques, including the key aspects of load balancing, mobile edge caching, and interference management. In both parts, appropriate state-of-the-art machine learning techniques (including probabilistic uncertainty cascades in proactive optimisation), complexity-performance trade-offs, and demonstrative examples are presented to inspire readers. This survey couples the potential of online big data analytics, cloud-edge computing, statistical machine learning, and proactive network optimisation in a common cross-layer wireless framework. The wider impact of this survey includes better cross-fertilising the academic fields of data analytics, mobile edge computing, AI, CPSS, and wireless communications, as well as informing the industry of the promising potentials in this area.INDEX TERMS Online data, data analytics, proactive network optimisation, 5G.

show abstract

Real time road traffic event detection using Twitter and spark

Cited by 25 publications

References 10 publications

Iktishaf+: A Big Data Tool with Automatic Labeling for Road Traffic Social Sensing and Event Detection Using Distributed Machine Learning

Iktishaf+: A Big Data Tool with Automatic Labeling for Road Traffic Social Sensing and Event Detection Using Distributed Machine Learning

Sehaa: A Big Data Analytics Tool for Healthcare Symptoms and Diseases Detection Using Twitter, Apache Spark, and Machine Learning

A Survey of Online Data-Driven Proactive 5G Network Optimisation Using Machine Learning

Contact Info

Product

Resources

About