Robust Truth Discovery against Data Poisoning in Mobile Crowdsensing

Huang, Zonghao; Pan, Miao; Gong, Yandao

doi:10.1109/globecom38437.2019.9013890

Cited by 11 publications

(6 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…24 The proposed method in this study is classified as a poisoning attack. Generally, attackers share some manipulated samples of data to be consumed by the learning algorithm when it is updating the model on new training data collected, for example, by crowdsensing or crowdsourcing systems from the physical world, 38,39 that, in turn, could push the learning quality to the worst level. The manipulated data can be of different formats such as images, 40,41 text, 42 or audio.…”

Section: Attacks Against MLmentioning

confidence: 99%

The robustness of popular multiclass machine learning models against poisoning attacks: Lessons and insights

Maabreh

Qolomany

et al. 2022

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

Despite the encouraging outcomes of machine learning and artificial intelligence applications, the safety of artificial intelligence–based systems is one of the most severe challenges that need further exploration. Data set poisoning is a severe problem that may lead to the corruption of machine learning models. The attacker injects data into the data set that are faulty or mislabeled by flipping the actual labels into the incorrect ones. The word “robustness” refers to a machine learning algorithm’s ability to cope with hostile situations. Here, instead of flipping the labels randomly, we use the clustering approach to choose the training samples for label changes to influence the classifiers’ performance and the distance-based anomaly detection capacity in quarantining the poisoned samples. According to our experiments on a benchmark data set, random label flipping may have a short-term negative impact on the classifier’s accuracy. Yet, an anomaly filter would discover on average 63% of them. On the contrary, the proposed clustering-based flipping might inject dormant poisoned samples until the number of poisoned samples is enough to influence the classifiers’ performance severely; on average, the same anomaly filter would discover 25% of them. We also highlight important lessons and observations during this experiment about the performance and robustness of popular multiclass learners against training data set–poisoning attacks that include: trade-offs, complexity, categories, poisoning resistance, and hyperparameter optimization.

show abstract

Section: Attacks Against MLmentioning

confidence: 99%

The robustness of popular multiclass machine learning models against poisoning attacks: Lessons and insights

Maabreh

Qolomany

et al. 2022

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

show abstract

“…The solution of data analysis usually focuses on the data analysis phase by using data processing methods such as Bayesian [ 18 ], machine learning clustering algorithm [ 19 ], and truth inference [ 20 , 21 ], in order to find the low quality of service. Lin et al [ 20 ] proposed a Sybil-resistant truth inference framework for MCS, which included three account grouping methods in pair with a truth inference algorithm to defend against the Sybil attacks.…”

Section: Related Workmentioning

confidence: 99%

“…Du et al [ 18 ] proposed a novel Bayesian co-clustering truth inference model, which produced an estimation while taking into account the entity clusters and the user clusters for the task of observation aggregation. Huang et al [ 21 ] investigated the data poisoning attacks on truth inference and proposed an approach against such attacks through additional source estimation and source filtering before data aggregation.…”

Section: Related Workmentioning

confidence: 99%

“…With the strategies S2–S3, spam workers may acquire a “good” reputation and then continuously pose threats to crowdsourcing platforms. According to previous research, it can be divided into four categories to improve the service quality: (1) Verification-based Defense Model [ 15 , 16 , 17 ]; (2) Data Analysis Solution [ 18 , 19 , 20 , 21 ]; (3) Workers’ Properties Matching Mechanism [ 22 , 23 , 24 , 25 ]; and (4) Trust-based Model [ 4 , 26 , 27 , 28 ].Verification-based Defense Models adopt the method of re-checking the submitted answers to discover spam workers so that these models require a lot of resources. Data Analysis Solutions filter out valuable services by analyzing data submitted by workers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

TruthTrust: Truth Inference-Based Trust Management Mechanism on a Crowdsourcing Platform

Zhou

Jin

et al. 2021

Sensors

View full text Add to dashboard Cite

On a crowdsourcing platform, in order to cheat for rewards or sabotage the crowdsourcing processes, spam workers may submit numerous erroneous answers to the tasks published by requesters. This type of behavior extremely reduces the completion rate of tasks and the enthusiasm of honest users, which may lead a crowdsourcing platform to a failure. Defending against malicious attacks is an important issue in crowdsourcing, which has been extensively addressed by existing methods, e.g., verification-based defense mechanisms, data analysis solutions, trust-based defense models, and workers’ properties matching mechanisms. However, verification-based defense mechanisms will consume a lot of resources, and data analysis solutions cannot motivate workers to provide high-quality services. Trust-based defense models and workers’ properties matching mechanisms cannot guarantee the authenticity of information when collusion requesters publish shadow tasks to help malicious workers get more participation opportunities. To defend such collusion attacks in crowdsourcing platforms, we propose a new defense model named TruthTrust. Firstly, we define a complete life cycle system that from users’ interaction to workers’ recommendation, and separately define the trust value of each worker and the credence of each requester. Secondly, in order to ensure the authenticity of the information, we establish a trust model based on the CRH framework. The calculated truth value and weight are used to define the global properties of workers and requesters. Moreover, we propose a reverse mechanism to improve the resistance under attacks. Finally, extensive experiments demonstrate that TruthTrust significantly outperforms the state-of-the-art approaches in terms of effective task completion rate.

show abstract

“…In smart cities, crowdsensing is an integral data source for smart services which is involved in several areas such as transportation, pollution monitoring, and energy management [114]. However, it is highly susceptible to data poisoning attacks [115], [116], and in some settings, gain greater degrees of reliability so that they are hard to be identified [117], [118]. In a very sensitive field of study, an experiment on around 17,000 records of healthy, unhealthy (disease-infected) people, a poisoning attack on the training data was able to drop the classifier accuracy of about 28% of its original accuracy by poising 30% of the data [104].…”

Section: Food Safetymentioning

confidence: 99%

Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges

Ahmad¹,

Maabreh²,

Ghaly³

et al. 2020

Preprint

View full text Add to dashboard Cite

As we make tremendous advances in machine learning and artificial intelligence (ML/AI) technosciences, there is a renewed understanding in the ML/AI community that we must ensure that humans being are at the center of our deliberations so that we don't end in technology-induced dystopias. As strongly argued by Green in his book "Smart Enough City", the incorporation of technology in city environs does not automatically translate into prosperity, wellbeing, urban livability, or social justice. There is a great need to deliberate on the future of the cities worth living and designing. There are philosophical and ethical questions involved along with various challenges that relate to the security, safety, and interpretability of ML/AI algorithms that will form the technological bedrock of future cities. A number of research institutes on humancentered AI (HAI) have been established at top international universities-including Stanford Institute for Human-Centered Artificial Intelligence (HAI), Berkeley's Center for Human-Compatible AI (CHAI), and MIT Institute for Data, Systems, and Society (IDSS). Globally there are calls for technology to be made more humane and human-compatible. For example, Stuart Russell has a book called Human-Compatible AI. The Center for Humane Technology advocates for regulators and technology companies to avoid business models and product features that contribute to social problems such as extremism, polarization, misinformation, and Internet addiction. In this paper, we analyze and explore key challenges including security, robustness, interpretability, and ethical (data and algorithmic) challenges to a successful deployment of AI/ML in human-centric applications, with a particular emphasis on the convergence of these concepts/challenges. We provide a detailed review of existing literature on these key challenges and analyze how one of these challenges may lead to others or help in solving other challenges. The paper also advises on the current limitations, pitfalls, and future directions of research in these domains, and how it can fill the current gaps and lead to better solutions. We believe such rigorous analysis will provide a baseline for future research in the domain.

show abstract

Robust Truth Discovery against Data Poisoning in Mobile Crowdsensing

Cited by 11 publications

References 32 publications

The robustness of popular multiclass machine learning models against poisoning attacks: Lessons and insights

The robustness of popular multiclass machine learning models against poisoning attacks: Lessons and insights

TruthTrust: Truth Inference-Based Trust Management Mechanism on a Crowdsourcing Platform

Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges

Contact Info

Product

Resources

About