Applying spark based machine learning model on streaming big data for health status prediction

Nair, Lekha R.; Shetty, Sujala D.; Shetty, Siddhanth D.

doi:10.1016/j.compeleceng.2017.03.009

Cited by 112 publications

(48 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The various benchmark datasets used in the past decade were WePS-3, 27 SemEval, 30,52,54,55,73,75,76,85 tweets prepared by Stanford University, 34,45,46,75 SNAP, 40 Sanders Twitter Sentiment Corpus (denoted as Sanders), 44,55,75,79 2008 Presidential Debate Corpus, 44,75,79 Sentiment140, 51 RepLab 2012, 53 RepLab 2013, 53 STS-manual, 55 Gold Standard personality labeled Twitter dataset, 59 Cleveland Heart Disease data, 69 STS-Gold, 73 FIGURE 6 Distribution of papers in accordance to the digital libraries (expressed in percentages) Many reported researches were carried on the tweets fetched directly from Twitter using its API. The tweets were from a variety of domains, topics and time period (referred as topic specific/topic oriented tweets).…”

Section: • Widely Used Datasets and Domains In Which The Studies For mentioning

confidence: 99%

Systematic literature review of sentiment analysis on Twitter using soft computing techniques

Kumar

Jaiswal

2019

Concurrency and Computation

133

View full text Add to dashboard Cite

Sentiment detection and classification is the latest fad for social analytics on Web. With the array of practical applications in healthcare, finance, media, consumer markets, and government, distilling the voice of public to gain insight to target information and reviews is non-trivial. With a marked increase in the size, subjectivity, and diversity of social web-data, the vagueness, uncertainty and imprecision within the information has increased manifold. Soft computing techniques have been used to handle this fuzziness in practical applications. This work is a study to understand the feasibility, scope and relevance of this alliance of using Soft computing techniques for sentiment analysis on Twitter. We present a systematic literature review to collate, explore, understand and analyze the efforts and trends in a well-structured manner to identify research gaps defining the future prospects of this coupling. The contribution of this paper is significant because firstly the primary focus is to study and evaluate the use of soft computing techniques for sentiment analysis on Twitter and secondly as compared to the previous reviews we adopt a systematic approach to identify, gather empirical evidence, interpret results, critically analyze, and integrate the findings of all relevant high-quality studies to address specific research questions pertaining to the defined research domain. KEYWORDS machine learning, review, sentiment analysis, soft computing, Twitter INTRODUCTIONThe incessantly evolving dynamics of the Web in terms of the volume, velocity and variety of opinion-rich information accessible online, has made research in the domain of Sentiment Analysis (SA) a trend for many practical applications which facilitate decision support and deliver targeted information to domain analysts. Interestingly, the buzzing term ''big data'' which is estimated to be 90% unstructured 1 further makes it crucial to tap and analyze information using contemporary tools. Text mining models define the process to transform and substitute this unstructured data into a structured one for knowledge discovery. Use of classification algorithms to intelligently mine text has been studied extensively across literature. 2,3 SA, established as a typical text classification task, 4 is defined as the computational study of people's opinions, attitudes and emotions towards an entity. 5,6 It offers a technology-based solution to understand people's reactions, views and opinion polarities (positive, negative or neutral) in textual content available over social media sources.Research studies and practical applications in the field of SA have escalated in the past decade with the transformation and expansion of Web from passive provider of content to an active socially-aware distributor of collective intelligence. This new collaborative Web (called Web 2.0), 7 extended by Web-based technologies like comments, blogs and wikis, social media portals like Twitter or Facebook, that allow to build social networks based on professional relationship, i...

show abstract

Section: • Widely Used Datasets and Domains In Which The Studies For mentioning

confidence: 99%

Systematic literature review of sentiment analysis on Twitter using soft computing techniques

Kumar

Jaiswal

2019

Concurrency and Computation

133

View full text Add to dashboard Cite

show abstract

“…Microblogging platforms, particularly Twitter, are valuable sources to extract sentiments of their users about movies, products, and events. In this manner, a real‐time SA may be used and integrated with recommendation systems or health status prediction system …”

Section: Real‐time Sentiment Prediction Frameworkmentioning

confidence: 99%

“…In this manner, a real-time SA may be used and integrated with recommendation systems or health status prediction system. 4,6 From a data point of perspective, social networking platforms generate Big Data that is huge in volume and in production velocity. Real-time analysis of such data requires new tools and techniques such as distributed computing frameworks.…”

Section: Real-time Sentiment Prediction Frameworkmentioning

confidence: 99%

“…An emergent solution to decrease this gap is to benefit from Big Data analytics solutions. There are mainly two Big Data processing strategies in SA, ie, (i) static strictly‐controlled structures, ie, batch processing, which stores data on a distributed file system and then uses a distributed computational framework such as Hadoop MapReduce (MR); and (ii) interactive or real time streaming data processing such as Apache Spark, which collects data as “streams” and processes the streaming data through an in‐memory computation strategy such as resilient distributed data sets (RDDs) …”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Kılınç

2019

Softw Pract Exp

View full text Add to dashboard Cite

Summary There are many data sources that produce large volumes of data. The Big Data nature requires new distributed processing approaches to extract the valuable information. Real‐time sentiment analysis is one of the most demanding research areas that requires powerful Big Data analytics tools such as Spark. Prior literature survey work has shown that, though there are many conventional sentiment analysis researches, there are only few works realizing sentiment analysis in real time. One major point that affects the quality of real‐time sentiment analysis is the confidence of the generated data. In more clear terms, it is a valuable research question to determine whether the owner that generates sentiment is genuine or not. Since data generated by fake personalities may decrease accuracy of the outcome, a smart/intelligent service that can identify the source of data is one of the key points in the analysis. In this context, we include a fake account detection service to the proposed framework. Both sentiment analysis and fake account detection systems are trained and tested using Naïve Bayes model from Apache Spark's machine learning library. The developed system consists of four integrated software components, ie, (i) machine learning and streaming service for sentiment prediction, (ii) a Twitter streaming service to retrieve tweets, (iii) a Twitter fake account detection service to assess the owner of the retrieved tweet, and (iv) a real‐time reporting and dashboard component to visualize the results of sentiment analysis. The sentiment classification performances of the system for offline and real‐time modes are 86.77% and 80.93%, respectively.

show abstract

“…The OAuth authentication has been used for security reasons, so that the data being sent is safe and secured. The data is then analyzed using the machine learning algorithm for heart disease prediction [11]. Machine learning is an effective computing technique that helps in prediction and classification.…”

Section: Literature Surveymentioning

confidence: 99%

Big data management with machine learning inscribed by domain knowledge for health care

Ephzibah

Sujatha

2017

IJET

View full text Add to dashboard Cite

In this work, a framework that helps in the disease diagnosis process with big-data management and machine learning using rule based, instance based, statistical, neural network and support vector method is given. Concerning this, big-data that contains the details of various diseases are collected, preprocessed and managed for classification. Diagnosis is a day-to-day activity for the medical practitioners and is also a decision-making task that requires domain knowledge and expertise in the specific field. This framework suggests different machine learning methods to aid the practitioner to diagnose disease based on the best classifier that is identified in the health care system. The framework has three main segments like big-data management, machine learning and input/output details of the patient. It has been already proved in the literature that the computing methods do help in disease diagnosis, provided the data about that particular disease is available in the data center. Thus this framework will provide a source of confidence and satisfaction to the doctors, as the model generated is based on the accuracy of the classifier compared to other classifiers.

show abstract

Applying spark based machine learning model on streaming big data for health status prediction

Cited by 112 publications

References 10 publications

Systematic literature review of sentiment analysis on Twitter using soft computing techniques

Systematic literature review of sentiment analysis on Twitter using soft computing techniques

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Big data management with machine learning inscribed by domain knowledge for health care

Contact Info

Product

Resources

About