CrowdCorrect: A Curation Pipeline for Social Data Cleansing and Curation

Beheshti, Amin; Vaghani, Kushal; Benatallah, Boualem; Tabebordbar, Alireza

doi:10.1007/978-3-319-92901-9_3

Cited by 25 publications

(18 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This website enables the usage of additional individuals and the payment of crowdsourcing fees, and the unrestricted use of personal contacts. Crowdsourcing necessitates stringent quality management or the use of trustworthy individuals 3,20,35 . Nonetheless, it will rapidly have links to many courts, thus improving multiple machine learning activities.…”

Section: Crowdsourcing As Post-processingmentioning

confidence: 99%

An Intelligent Hybrid Model for Identity Document Classification

Khandan

2021

Preprint

View full text Add to dashboard Cite

Digitization, i.e., the process of converting information into a digital format, may provide various opportunities (e.g., increase in productivity, disaster recovery, and environmentally friendly solutions) and challenges for businesses. In this context, one of the main challenges would be to accurately classify numerous scanned documents uploaded every day by customers as usual business processes. For example, processes in banking (e.g., applying for loans) or the Government Registry of BDM (Births, Deaths, and Marriages) applications may involve uploading several documents such as a driver's license and passport. There are not many studies available to address the challenge as an application of image classification. Although some studies are available which used various methods, a more accurate model is still required. The current study has proposed a robust fusion model to define the type of identity documents accurately. The proposed approach is based on two different methods in which images are classified based on their visual features and text features. A novel model based on statistics and regression has been proposed to calculate the confidence level for the feature-based classifier. A fuzzy-mean fusion model has been proposed to combine the classifier results based on their confidence score. The proposed approach has been implemented using Python and experimentally validated on synthetic and real-world datasets. The performance of the proposed model is evaluated using the Receiver Operating Characteristic (ROC) curve analysis.

show abstract

Section: Crowdsourcing As Post-processingmentioning

confidence: 99%

An Intelligent Hybrid Model for Identity Document Classification

Khandan

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Social media data needs to be curated before being used for deeper analytics. A data curation pipeline, named Crowd-Correct, was proposed by Beheshti et al (2018) that exploits the hybrid combination of machine and human-driven functionality to curate and clean social media data. The procedures of automatic feature extraction, correction and enrichment are performed in the first step.…”

Section: Expert-based Mechanismsmentioning

confidence: 99%

Data cleansing mechanisms and approaches for big data analytics: a systematic study

Hosseinzadeh

Azhir

Ahmed

et al. 2021

J Ambient Intell Human Comput

View full text Add to dashboard Cite

With the evolution of new technologies, the production of digital data is constantly growing. It is thus necessary to develop data management strategies in order to handle the large-scale datasets. The data gathered through different sources, such as sensor networks, social media, business transactions, etc. is inherently uncertain due to noise, missing values, inconsistencies and other problems that impact the quality of big data analytics. One of the key challenges in this context is to detect and repair dirty data, i.e. data cleansing, and various techniques have been presented to solve this issue. However, to the best of our knowledge, there has not been any comprehensive review of data cleansing techniques for big data analytics. As such, a comprehensive and systematic study on the state-of-the-art mechanisms within the scope of the big data cleansing is done in this survey. Therefore, five categories to review these mechanisms are considered, which are machine learningbased, sample-based, expert-based, rule-based, and framework-based mechanisms. A number of articles are reviewed in each category. Furthermore, this paper denotes the advantages and disadvantages of the chosen data cleansing techniques and discusses the related parameters, comparing them in terms of scalability, efficiency, accuracy, and usability. Finally, some suggestions for further work are provided to improve the big data cleansing mechanisms in the future.

show abstract

“…They proposed an integration of APIs which facilitate the use of features to assists users in achieving the desired curated data. In other studies, Behesti et al 33,34 developed a curation pipeline for data from social networks. They also highlight the importance of data cleaning to support the curation process.…”

Section: Related Workmentioning

confidence: 99%

Data curation in the Internet of Things: A decision model approach

Haro-Olmo

Valencia-Parra

Varela-Vaca

et al. 2021

Comp and Math Methods

View full text Add to dashboard Cite

Current Internet of Things (IoT) scenarios have to deal with many challenges especially when a large amount of heterogeneous data sources are integrated, that is, data curation. In this respect, the use of poor‐quality data (i.e., data with problems) can produce terrible consequence from incorrect decision‐making to damaging the performance in the operations. Therefore, using data with an acceptable level of usability has become essential to achieve success. In this article, we propose an IoT‐big data pipeline architecture that enables data acquisition and data curation in any IoT context. We have customized the pipeline by including the DMN4DQ approach to enable us the measuring and evaluating data quality in the data produced by IoT sensors. Further, we have chosen a real dataset from sensors in an agricultural IoT context and we have defined a decision model to enable us the automatic measuring and assessing of the data quality with regard to the usability of the data in the context.

show abstract

CrowdCorrect: A Curation Pipeline for Social Data Cleansing and Curation

Cited by 25 publications

References 30 publications

An Intelligent Hybrid Model for Identity Document Classification

An Intelligent Hybrid Model for Identity Document Classification

Data cleansing mechanisms and approaches for big data analytics: a systematic study

Data curation in the Internet of Things: A decision model approach

Contact Info

Product

Resources

About