A transient search using combined human and machine classifications

Wright, D. E.; Lintott, Chris; Smartt, S. J.; Smith, K.; Fortson, L.; Trouille, Laura; Allen, Campbell; Beck, Mélanie; Bouslog, Mark C.; Boyer, Amy; Chambers, K.; Flewelling, H.; Granger, Will; Magnier, E. A.; McMaster, Adam; Miller, Grant; O’Donnell, Jim; Simmons, Brooke; Spiers, Helen; Tonry, J.; Veldthuis, Marten; Wainscoat, R. J.; Waters, C.; Willman, M.; Wolfenbarger, Zach; Young, D. R.

doi:10.1093/mnras/stx1812

Cited by 46 publications

(26 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other astrophysics research has combined crowdsourcing with machine learning models. Wright et al (2017) classified supernovae in PanSTARRS (Kaiser et al 2010) by aggregating crowdsourced classifications with the predictions of expert-trained CNN and show that the combined human/machine ensemble outperforms either alone. However, this approach is not directly feasible for Galaxy Zoo, where scale prevents us from recording crowdsourced classifications for every image.…”

Section: Active Learning Approach For Galaxy Zoomentioning

confidence: 99%

Galaxy Zoo: probabilistic morphology through Bayesian CNNs and active learning

Walmsley

Smith

Lintott

et al. 2019

Monthly Notices of the Royal Astronomical Society

Self Cite

135

View full text Add to dashboard Cite

We use Bayesian convolutional neural networks and a novel generative model of Galaxy Zoo volunteer responses to infer posteriors for the visual morphology of galaxies. Bayesian CNN can learn from galaxy images with uncertain labels and then, for previously unlabelled galaxies, predict the probability of each possible label. Our posteriors are well-calibrated (e.g. for predicting bars, we achieve coverage errors of 10.6% within 5 responses and 2.9% within 10 responses) and hence are reliable for practical use. Further, using our posteriors, we apply the active learning strategy BALD to request volunteer responses for the subset of galaxies which, if labelled, would be most informative for training our network. We show that training our Bayesian CNNs using active learning requires up to 35-60% fewer labelled galaxies, depending on the morphological feature being classified. By combining human and machine intelligence, Galaxy Zoo will be able to classify surveys of any conceivable scale on a timescale of weeks, providing massive and detailed morphology catalogues to support research into galaxy evolution.

show abstract

Section: Active Learning Approach For Galaxy Zoomentioning

confidence: 99%

Galaxy Zoo: probabilistic morphology through Bayesian CNNs and active learning

Walmsley

Smith

Lintott

et al. 2019

Monthly Notices of the Royal Astronomical Society

Self Cite

135

View full text Add to dashboard Cite

show abstract

“…Finally, citizen science classifications can be used to train future machine learning algorithms, and the counts produced by each can be cross-validated, ensuring the analyses remain reliable. In fact, Wright et al (2017) demonstrate that a combination of methods (i.e. citizen science and machine learning) can outperform either method used alone 20 .…”

Section: Methodsmentioning

confidence: 96%

“…In fact, Wright et al (2017) demonstrate that a combination of methods (i.e. citizen science and machine learning) can outperform either method used alone 20 .…”

Section: Methodsmentioning

confidence: 96%

Processing citizen science- and machine-annotated time-lapse imagery for biologically meaningful metrics

et al. 2020

Self Cite

View full text Add to dashboard Cite

time-lapse cameras facilitate remote and high-resolution monitoring of wild animal and plant communities, but the image data produced require further processing to be useful. Here we publish pipelines to process raw time-lapse imagery, resulting in count data (number of penguins per image) and 'nearest neighbour distance' measurements. the latter provide useful summaries of colony spatial structure (which can indicate phenological stage) and can be used to detect movement-metrics which could be valuable for a number of different monitoring scenarios, including image capture during aerial surveys. We present two alternative pathways for producing counts: (1) via the Zooniverse citizen science project Penguin Watch and (2) via a computer vision algorithm (Pengbot), and share a comparison of citizen science-, machine learning-, and expert-derived counts. We provide example files for 14 Penguin Watch cameras, generated from 63,070 raw images annotated by 50,445 volunteers. We encourage the use of this large open-source dataset, and the associated processing methodologies, for both ecological studies and continued machine learning and computer vision development.

show abstract

“…Ref. 54 found that the human classifications and machine-learning results in Pan-STARRS were complementary; the human classifications provide a pure but incomplete sample while the machine classifications provide a complete but impure sample. By combining the aggregated volunteer classifications with the machine-learning results, they are able to create the most pure and complete sample of new supernovae candidates.…”

Section: Experiments In Machine Integrationmentioning

confidence: 99%

Citizen science frontiers: Efficiency, engagement, and serendipitous discovery with human–machine systems

Trouille

Lintott

Fortson

2019

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

Citizen science has proved to be a unique and effective tool in helping science and society cope with the ever-growing data rates and volumes that characterize the modern research landscape. It also serves a critical role in engaging the public with research in a direct, authentic fashion and by doing so promotes a better understanding of the processes of science. To take full advantage of the onslaught of data being experienced across the disciplines, it is essential that citizen science platforms leverage the complementary strengths of humans and machines. ThisPerspectivespiece explores the issues encountered in designing human–machine systems optimized for both efficiency and volunteer engagement, while striving to safeguard and encourage opportunities for serendipitous discovery. We discuss case studies from Zooniverse, a large online citizen science platform, and show that combining human and machine classifications can efficiently produce results superior to those of either one alone and how smart task allocation can lead to further efficiencies in the system. While these examples make clear the promise of human–machine integration within an online citizen science system, we then explore in detail how system design choices can inadvertently lower volunteer engagement, create exclusionary practices, and reduce opportunity for serendipitous discovery. Throughout we investigate the tensions that arise when designing a human–machine system serving the dual goals of carrying out research in the most efficient manner possible while empowering a broad community to authentically engage in this research.

show abstract

A transient search using combined human and machine classifications

Cited by 46 publications

References 36 publications

Galaxy Zoo: probabilistic morphology through Bayesian CNNs and active learning

Galaxy Zoo: probabilistic morphology through Bayesian CNNs and active learning

Processing citizen science- and machine-annotated time-lapse imagery for biologically meaningful metrics

Citizen science frontiers: Efficiency, engagement, and serendipitous discovery with human–machine systems

Contact Info

Product

Resources

About