Classification of multiple annotator data using variational Gaussian process inference

Besler, Emre; Ruiz, Pablo; Molina, Rafael; Katsaggelos, Aggelos K.

doi:10.1109/eusipco.2016.7760604

Cited by 3 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, to model the underlying real labels z given the features X, Gaussian Processes (GP) has proven to be the most successful probabilistic approach, mainly because of its great flexibility and excellent uncertainty quantification [13,21,22]…”

Section: Probabilistic Modellingmentioning

confidence: 99%

“…Then, given each latent variable f n , the underlying real label z n is modelled with the sigmoid function σ, p(z n = 1|f n ) = σ(f n ) = (1 + exp(−f n )) −1 . Under this common classical model, the main difference between the previous approaches [13] and [21,22] is the inference procedure used: Expectation Propagation [32] in the former and Variational Inference [19,20] in the latter (recall the second paragraph of Section 1). Figure 1a) shows a graphical representation of this GP-based classical model, which is in the basis of our proposal.…”

Section: Probabilistic Modellingmentioning

confidence: 99%

“…Expectation Propagation (EP) [18] (see also [14,Section 3.6]) was used as inference procedure for GP in [13]. Recently, Variational Inference (VI) [19,20] was used as an alternative to EP in crowdsourcing, outperforming it in both predictive performance and computational cost [21,22]. These probabilistic GP-based methods have proven very successful in the crowdsourcing literature.…”

Section: Introductionmentioning

confidence: 99%

“…In this work, we start by applying the aforementioned Fourier features methodology to approximate the squared exponential (SE) kernel of the GPbased crowdsourcing method proposed in [21,22]. This approach is referred to as RFF (Random Fourier Features).…”

Section: Introductionmentioning

confidence: 99%

“…These are very significant reductions with respect to previous approaches. Whereas RFF is a large-scale approximation of the previous approach in [21,22], VFF is a whole new scalable crowdsourcing method, whose additional flexibility allows one to capture new relevant patterns (even in previously-reachable small datasets). However, VFF is more prone to overfitting, and slower in practice.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Scalable and efficient learning from crowds with Gaussian processes

et al. 2019

Self Cite

View full text Add to dashboard Cite

Over the last few years, multiply-annotated data has become a very popular source of information. Online platforms such as Amazon Mechanical Turk have revolutionized the labelling process needed for any classification task, sharing the effort between a number of annotators (instead of the classical single expert). This crowdsourcing approach has introduced new challenging problems, such as handling disagreements on the annotated samples or combining the unknown expertise of the annotators. Probabilistic methods, such as Gaussian Processes (GP), have proven successful to model this new crowdsourcing scenario. However, GPs do not scale up well with the training set size, which makes them prohibitive for medium-to-large datasets (beyond 10K training instances). This constitutes a serious limitation for current real-world applications. In this work, we introduce two scalable and efficient GP-based crowdsourcing methods that allow for processing previously-prohibitive datasets. The first one is an efficient and fast approximation to GP with squared exponential (SE) kernel. The second allows for learning a more flexible kernel at the expense of a heavier training (but still scalable to large datasets). Since the latter is not a GP-SE approximation, it can be also considered as a whole new scalable and efficient crowdsourcing method, useful for any dataset size. Both methods use Fourier features and variational inference, can predict the class of new samples, and estimate the expertise of the involved annotators. A complete experimentation compares them with state-of-the-art probabilistic approaches in synthetic and real crowdsourcing datasets of different sizes. They stand out as the best performing approach for large scale problems. Moreover, the second method is competitive with the current state-of-the-art for small datasets.

show abstract

Section: Probabilistic Modellingmentioning

confidence: 99%

Section: Probabilistic Modellingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Scalable and efficient learning from crowds with Gaussian processes

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

Gravity Spy: integrating advanced LIGO detector characterization, machine learning, and citizen science

Zevin

Coughlin

Bahaadini

et al. 2017

Class. Quantum Grav.

307

347

View full text Add to dashboard Cite

Abstract. With the first direct detection of gravitational waves, the Advanced Laser Interferometer Gravitational-wave Observatory (LIGO) has initiated a new field of astronomy by providing an alternate means of sensing the universe. The extreme sensitivity required to make such detections is achieved through exquisite isolation of all sensitive components of LIGO from non-gravitational-wave disturbances. Nonetheless, LIGO is still susceptible to a variety of instrumental and environmental sources of noise that contaminate the data. Of particular concern are noise features known as glitches, which are transient and non-Gaussian in their nature, and occur at a high enough rate so that accidental coincidence between the two LIGO detectors is non-negligible. Glitches come in a wide range of time-frequency-amplitude morphologies, with new morphologies appearing as the detector evolves. Since they can obscure or mimic true gravitational-wave signals, a robust characterization of glitches is paramount in the effort to achieve the gravitational-wave detection rates that are predicted by the design sensitivity of LIGO. This proves a daunting task for members of the LIGO Scientific Collaboration alone due to the sheer amount of data. In this paper we describe an innovative project that combines crowdsourcing with machine learning to aid in the challenging task of categorizing all of the glitches recorded by the LIGO detectors. Through the Zooniverse platform, we engage and recruit volunteers from the public to categorize images of time-frequency representations of glitches into preidentified morphological classes and to discover new classes that appear as the detectors arXiv:1611.04596v2 [gr-qc]

show abstract