Anomaly detection in the Zwicky Transient Facility DR3

Abstract

We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of 3 stages: feature extraction, search of outliers with machine learning algorithms and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million objects. A set of 4 automatic learning algorithms was used to identify 277 outliers, which were subsequently scrutinised by an expert. From these, 188 (68%) were found to be bogus light curves – including effects from the image subtraction pipeline as well as overlapping between a star and a known asteroid, 66 (24%) were previously reported sources whereas 23 (8%) correspond to non-catalogued objects, with the two latter cases of potential scientific interest (e. g. 1 spectroscopically confirmed RS Canum Venaticorum star, 4 supernovae candidates, 1 red dwarf flare). Moreover, using results from the expert analysis, we were able to identify a simple bi-dimensional relation which can be used to aid filtering potentially bogus light curves in future studies. We provide a complete list of objects with potential scientific application so they can be further scrutinised by the community. These results confirm the importance of combining automatic machine learning algorithms with domain knowledge in the construction of recommendation systems for astronomy. Our code is publicly available*.

Real-time Detection of Anomalies in Multivariate Time Series of Astronomical Data

“…This work and other recent approaches to transient anomaly detection [e.g. 16,18,[20][21][22][23] are going to be critical for discovery in the new era of large-scale astronomical surveys.…”

Searching for Anomalies in the ZTF Catalog of Periodic Variable Stars

“…Ichinohe & Yamada (2019) suggested searching for anomalous X-ray transients using a variational autoencoder. Malanchev et al (2021) extracted features from light curves given by the ZTF Data Release 3 and searched for anomalies using the isolation forest, Local Outlier Factor, Gaussian Mixture Model, and One-class Support Vector Machines. From this search, they identified 23 non-cataloged astrophysical events of interest.…”

“…Previous studies on periodic variables that utilize machine learning focus primarily on classifications (Jamal & Bloom 2020;Zhang & Bloom 2021;Naul et al 2018), and deep generative modeling (Martínez-Palomera et al 2020). Although Malanchev et al (2021) searched for anomalous transient detected with ZTF, they did not specifically focus on periodic variables. Here, we aim to provide an anomaly detection algorithm to effectively search for anomalous periodic variables detected with ZTF.…”

SNAD Transient Miner: Finding Missed Transient Events in ZTF DR4 using k-D trees

“…The SNAD 2 team has been continuously working in the development of anomaly detection algorithms which are able to prove their efficiency in real data while incorporating domain knowledge in the machine learning model -thus tailoring it according to the scientific interest of the expert (e.g., Pruzhinskaya et al, 2019;Aleo et al, 2020;Malanchev et al, 2021;Ishida et al, 2021). In this work, we present a hybrid approach for mining transients in large astronomical datasets, specifically ZTF DR4; moreover, our methodology can also be applied to the nightly ZTF alert-stream via timedomain brokers like ANTARES (Matheson et al, 2021) and FINK (Möller et al, 2021).…”