Automatic Detection and Validation of Smart City Events Using HPC and Apache Spark Platforms

Suma, Sugimiyanto; Mehmood, Rashid; Albeshri, Aiiad

doi:10.1007/978-3-030-13705-2_3

Cited by 21 publications

(20 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Earlier work on this tool has focused mainly on mobility-related event detection using supervised learning. We have also developed other tools for big data social media analytics in healthcare [ 10 ], logistics [ 11 , 12 ], and public opinion mining for government services [ 13 ]. These works have used Twitter data in Arabic or English.…”

Section: Introductionmentioning

confidence: 99%

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

Alomari

Katib

Albeshri

et al. 2021

IJERPH

Self Cite

View full text Add to dashboard Cite

Today’s societies are connected to a level that has never been seen before. The COVID-19 pandemic has exposed the vulnerabilities of such an unprecedently connected world. As of 19 November 2020, over 56 million people have been infected with nearly 1.35 million deaths, and the numbers are growing. The state-of-the-art social media analytics for COVID-19-related studies to understand the various phenomena happening in our environment are limited and require many more studies. This paper proposes a software tool comprising a collection of unsupervised Latent Dirichlet Allocation (LDA) machine learning and other methods for the analysis of Twitter data in Arabic with the aim to detect government pandemic measures and public concerns during the COVID-19 pandemic. The tool is described in detail, including its architecture, five software components, and algorithms. Using the tool, we collect a dataset comprising 14 million tweets from the Kingdom of Saudi Arabia (KSA) for the period 1 February 2020 to 1 June 2020. We detect 15 government pandemic measures and public concerns and six macro-concerns (economic sustainability, social sustainability, etc.), and formulate their information-structural, temporal, and spatio-temporal relationships. For example, we are able to detect the timewise progression of events from the public discussions on COVID-19 cases in mid-March to the first curfew on 22 March, financial loan incentives on 22 March, the increased quarantine discussions during March–April, the discussions on the reduced mobility levels from 24 March onwards, the blood donation shortfall late March onwards, the government’s 9 billion SAR (Saudi Riyal) salary incentives on 3 April, lifting the ban on five daily prayers in mosques on 26 May, and finally the return to normal government measures on 29 May 2020. These findings show the effectiveness of the Twitter media in detecting important events, government measures, public concerns, and other information in both time and space with no earlier knowledge about them.

show abstract

Section: Introductionmentioning

confidence: 99%

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

Alomari

Katib

Albeshri

et al. 2021

IJERPH

Self Cite

View full text Add to dashboard Cite

show abstract

“…Social media could be very effective in detecting various events (such as pandemics), understanding the effectiveness of policies and procedures related to pandemic management, understating public views on various matters, etc. [82,83,[208][209][210][211][212]. Monitoring and tracing people and their activities are important; however, related privacy and security concerns must also be addressed [196,197].…”

Section: Discussionmentioning

confidence: 99%

iResponse: An AI and IoT-Enabled Framework for Autonomous COVID-19 Pandemic Management

et al. 2021

Self Cite

View full text Add to dashboard Cite

SARS-CoV-2, a tiny virus, is severely affecting the social, economic, and environmental sustainability of our planet, causing infections and deaths (2,674,151 deaths, as of 17 March 2021), relationship breakdowns, depression, economic downturn, riots, and much more. The lessons that have been learned from good practices by various countries include containing the virus rapidly; enforcing containment measures; growing COVID-19 testing capability; discovering cures; providing stimulus packages to the affected; easing monetary policies; developing new pandemic-related industries; support plans for controlling unemployment; and overcoming inequalities. Coordination and multi-term planning have been found to be the key among the successful national and global endeavors to fight the pandemic. The current research and practice have mainly focused on specific aspects of COVID-19 response. There is a need to automate the learning process such that we can learn from good and bad practices during pandemics and normal times. To this end, this paper proposes a technology-driven framework, iResponse, for coordinated and autonomous pandemic management, allowing pandemic-related monitoring and policy enforcement, resource planning and provisioning, and data-driven planning and decision-making. The framework consists of five modules: Monitoring and Break-the-Chain, Cure Development and Treatment, Resource Planner, Data Analytics and Decision Making, and Data Storage and Management. All modules collaborate dynamically to make coordinated and informed decisions. We provide the technical system architecture of a system based on the proposed iResponse framework along with the design details of each of its five components. The challenges related to the design of the individual modules and the whole system are discussed. We provide six case studies in the paper to elaborate on the different functionalities of the iResponse framework and how the framework can be implemented. These include a sentiment analysis case study, a case study on the recognition of human activities, and four case studies using deep learning and other data-driven methods to show how to develop sustainability-related optimal strategies for pandemic management using seven real-world datasets. A number of important findings are extracted from these case studies.

show abstract

“…The challenges in this respect include management, integration, and distributed computation of data including the difficulties related to managing the 4V characteristics of big data, i.e., volume, velocity, variety, and veracity. There have been some works on the use of big data platforms in Twitter data analytics in various languages but in different application domains [18][19][20][21][22][23]37,78,79]. To the best of our knowledge, no work has been reported that uses big data technologies for data analytics in healthcare using tweets in the Arabic language.…”

Section: Research Gapmentioning

confidence: 99%

Sehaa: A Big Data Analytics Tool for Healthcare Symptoms and Diseases Detection Using Twitter, Apache Spark, and Machine Learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

Smartness, which underpins smart cities and societies, is defined by our ability to engage with our environments, analyze them, and make decisions, all in a timely manner. Healthcare is the prime candidate needing the transformative capability of this smartness. Social media could enable a ubiquitous and continuous engagement between healthcare stakeholders, leading to better public health. Current works are limited in their scope, functionality, and scalability. This paper proposes Sehaa, a big data analytics tool for healthcare in the Kingdom of Saudi Arabia (KSA) using Twitter data in Arabic. Sehaa uses Naive Bayes, Logistic Regression, and multiple feature extraction methods to detect various diseases in the KSA. Sehaa found that the top five diseases in Saudi Arabia in terms of the actual afflicted cases are dermal diseases, heart diseases, hypertension, cancer, and diabetes. Riyadh and Jeddah need to do more in creating awareness about the top diseases. Taif is the healthiest city in the KSA in terms of the detected diseases and awareness activities. Sehaa is developed over Apache Spark allowing true scalability. The dataset used comprises 18.9 million tweets collected from November 2018 to September 2019. The results are evaluated using well-known numerical criteria (Accuracy and F1-Score) and are validated against externally available statistics.

show abstract

Automatic Detection and Validation of Smart City Events Using HPC and Apache Spark Platforms

Cited by 21 publications

References 39 publications

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

iResponse: An AI and IoT-Enabled Framework for Autonomous COVID-19 Pandemic Management

Sehaa: A Big Data Analytics Tool for Healthcare Symptoms and Diseases Detection Using Twitter, Apache Spark, and Machine Learning

Contact Info

Product

Resources

About