This paper addresses the problem of extracting accurate labels from crowdsourced datasets, a key challenge in crowdsourcing. Prior work has focused on modeling the reliability of individual workers, for instance, by way of confusion matrices, and using these latent traits to estimate the true labels more accurately. However, this strategy becomes ineffective when there are too few labels per worker to reliably estimate their quality. To mitigate this issue, we propose a novel community-based Bayesian label aggregation model, CommunityBCC, which assumes that crowd workers conform to a few different types, where each type represents a group of workers with similar confusion matrices. We assume that each worker belongs to a certain community, where the worker's confusion matrix is similar to (a perturbation of) the community's confusion matrix. Our model can then learn a set of key latent features: (i) the confusion matrix of each community, (ii) the community membership of each user, and (iii) the aggregated label of each item. We compare the performance of our model against established aggregation methods on a number of large-scale, real-world crowdsourcing datasets. Our experimental results show that our CommunityBCC model consistently outperforms stateof-the-art label aggregation methods, gaining, on average, 8% more accuracy with the same amount of labels.
Abstract. The implausibility of the extreme rationality assumptions of Nash equilibrium has been attested by numerous experimental studies with human players. In particular, the fundamental social dilemmas such as the Traveler's dilemma, the Prisoner's dilemma, and the Public Goods game demonstrate high rates of deviation from the unique Nash equilibrium, dependent on the game parameters or the environment in which the game is played. These results inspired several attempts to develop suitable solution concepts to more accurately explain human behaviour. In this line, the recently proposed notion of cooperative equilibrium, [5], [6], based on the idea that players have a natural attitude to cooperation, has shown promising results for single-shot games. In this paper, we extend this approach to iterated settings. Specifically, we define the Iterated Cooperative Equilibrium (ICE) and show it makes statistically precise predictions of population average behaviour in the aforementioned domains. Importantly, the definition of ICE does not involve any free parameters, and so it is fully predictive.
The use of system modelling tools for design and transient analysis of LRE is becoming more and more common since their accuracy is constantly improving. But for this kind of tools prediction of film cooling effectiveness is still a challenging problem. A film cooling model capable of describing the benefits of such a cooling technique, as well as the impacts on the propulsion system such as loss in performance and requirements induced on other subsystems, can support the preliminary design of LRE systems.EcosimPro is an object oriented tool capable of modelling various kinds of dynamic systems. The model described within this paper is implemented alongside ESPSS 7 the propulsion system library compatible with EcosimPro. This paper covers a Quasi 2-D integral formulation to study the developed flow field of a film wall jet in combustion chambers. With this approach it is possible to have a fast prediction of the evolution in space and time of both coolant and wall temperature distribution therefore also of the film cooling effectiveness.A new model is here presented: a "layered model" based on the assumption that the mixing zone does not affect the film and mainstream flows but it is a result of their interaction. The geometry of the developed flow field is based on the one carried out by Simon 12 and it accounts for the core, mixing and film regions. The governing equations already present in the original combustion chamber model are used as a starting point. The implementation here described can easily be embedded in an unsteady propulsion system analysis for simulating the its transient phases and steady state operation points. The code flexibility allows for using the same model also for performance analysis in steady-state, and for off-design studies. First validation results are given here, in particular a DLR H2/O2 combustion chamber film cooling test campaign has been taken as reference case for validation purposes. NomenclatureA Area, m 2 b Wall jet thickness, m C m Mixing coefficient, -E Energy per unit of mass, J/kg F (R) Jet interface position function, -h Flow total enthalpy, kJ/kg h c Convective heat transfer coefficient, W/(m 2 ·K) M Blowing ratio, ρs vs ρ∞ v∞ , -m Mass flow rate, kg/s P Static pressure, baṙ Q Heat flow , Ẇ q Heat flux density, W/m 2 R Velocity ratio, v∞ vs , -S Film injection slot height, m St Stanton number, -T Temperature, K v Flow velocity, m/s x Distance downstream of the slot, m
Many aspects of the design of efficient crowdsourcing processes, such as defining worker’s bonuses, fair prices and time limits of the tasks, involve knowledge of the likely duration of the task at hand. In this work we introduce a new time–sensitive Bayesian aggregation method that simultaneously estimates a task’s duration and obtains reliable aggregations of crowdsourced judgments. Our method, called BCCTime, uses latent variables to represent the uncertainty about the workers’ completion time, the tasks’ duration and the workers’ accuracy. To relate the quality of a judgment to the time a worker spends on a task, our model assumes that each task is completed within a latent time window within which all workers with a propensity to genuinely attempt the labelling task (i.e., no spammers) are expected to submit their judgments. In contrast, workers with a lower propensity to valid labelling, such as spammers, bots or lazy labellers, are assumed to perform tasks considerably faster or slower than the time required by normal workers. Specifically, we use efficient message-passing Bayesian inference to learn approximate posterior probabilities of (i) the confusion matrix of each worker, (ii) the propensity to valid labelling of each worker, (iii) the unbiased duration of each task and (iv) the true label of each task. Using two realworld public datasets for entity linking tasks, we show that BCCTime produces up to 11% more accurate classifications and up to 100% more informative estimates of a task’s duration compared to state–of–the–art methods
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.