Dynamic Truth Discovery on Numerical Data

Zhi, Shi; Yang, Fan; Zhu, Zheyi; Li, Qi; Wang, Zhaoran

doi:10.1109/icdm.2018.00097

Cited by 20 publications

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dynamic truth discovery problem has also received a lot of attention in recent years. Zhi et al [23] proposed using first-order Markov process to represent the temporal dynamics of time series. Observations are represented by hidden Markov model, in which the truths are considered as the latent variables.…”

Section: Related Work a Time Series Truth Discoverymentioning

confidence: 99%

Finding the Truth From Uncertain Time Series by Differencing

Sun,

Zhou,

Jiang

2023

IEEE Open J. Comput. Soc.

View full text Add to dashboard Cite

Time series data is ubiquitous and of great importance in real applications. But due to poor qualities and bad working conditions of sensors, time series reported by them contain more or less noises. To reduce noise, multiple sensors are usually deployed to measure an identical time series and from these observations the truth can be estimated, which derives the problem of truth discovery for uncertain time series data. Several algorithms have been proposed, but they mainly focus on minimizing the error between the estimated truth and the observations. In our study, we aim at minimizing the noise in the estimated truth. To solve this optimization problem, we first find out the level of noise produced by each sensor based on differenced time series, which can help estimating the truth wisely. Then, we propose a quadratic optimization model to minimize the noise of the estimated truth. Further, a post process is introduced to refine the result by iteration. Experimental results on both real world and synthetic data sets verify the effectiveness and efficiency of our proposed methods respectively.

show abstract

Section: Related Work a Time Series Truth Discoverymentioning

confidence: 99%

Finding the Truth From Uncertain Time Series by Differencing

Sun,

Zhou,

Jiang

2023

IEEE Open J. Comput. Soc.

View full text Add to dashboard Cite

show abstract

“…In the aforementioned methods, the instances are assumed to be independent. More recently, methods are developed to handle various types of correlations among annotation instances such as spatial-temporal dependencies among instances [24], [48], [53]. Recent survey papers [21], [52] have shown that the aggregation methods which consider the worker reliabilities significantly outperform the naive aggregation methods such as voting.…”

Section: Related Workmentioning

confidence: 99%

Truth Discovery in Sequence Labels from Crowds

Sabetpour¹,

Kulkarni²,

Xie³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Annotations quality and quantity positively affect the performance of sequence labeling, a vital task in Natural Language Processing. Hiring domain experts to annotate a corpus set is very costly in terms of money and time. Crowdsourcing platforms, such as Amazon Mechanical Turk (AMT), have been deployed to assist in this purpose. However, these platforms are prone to human errors due to the lack of expertise; hence, one worker's annotations cannot be directly used to train the model. Existing literature in annotation aggregation more focuses on binary or multi-choice problems. In recent years, handling the sequential label aggregation tasks on imbalanced datasets with complex dependencies between tokens has been challenging. To conquer the challenge, we propose an optimization-based method that infers the best set of aggregated annotations using labels provided by workers. The proposed Aggregation method for Sequential Labels from Crowds (AggSLC) jointly considers the characteristics of sequential labeling tasks, workers' reliabilities, and advanced machine learning techniques. We evaluate AggSLC on different crowdsourced data for Named Entity Recognition (NER), Information Extraction tasks in biomedical (PICO), and the simulated dataset. Our results show that the proposed method outperforms the state-of-the-art aggregation methods. To achieve insights into the framework, we study AggSLC components' effectiveness through ablation studies by evaluating our model in the absence of the prediction module and inconsistency loss function. Theoretical analysis of our algorithm's convergence points that the proposed AggSLC halts after a finite number of iterations.

show abstract