A novel single-pulse search approach to detection of dispersed radio pulses using clustering and supervised machine learning

Pang, Di; Goševa-Popstojanova, Katerina; Devine, Thomas; McLaughlin, M. A.

doi:10.1093/mnras/sty1992

Cited by 26 publications

(43 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A bright pulse will be detected optimally in the DM trial and time bin most closely matching the true event, but also in other nearby DM trials and possibly in multiple matched filter trials. Grouping can be performed with a friends-of-friends algorithm that searches for clusters of points in a specified parameter space 1213 (Pang et al, 2018). Alternatively, an acceptable proximity margin can be specified and two candidates within that margin are grouped together.…”

Section: Candidate Groupingmentioning

confidence: 99%

Fast radio bursts

Petroff

Hessels

Lorimer

2019

Astron Astrophys Rev

622

471

View full text Add to dashboard Cite

The discovery of radio pulsars over a half century ago was a seminal moment in astronomy. It demonstrated the existence of neutron stars, gave a powerful observational tool to study them, and has allowed us to probe strong gravity, dense matter, and the interstellar medium. More recently, pulsar surveys have led to the serendipitous discovery of fast radio bursts (FRBs). While FRBs appear similar to the individual pulses from pulsars, their large dispersive delays suggest that they originate from far outside the Milky Way and hence are many ordersof-magnitude more luminous. While most FRBs appear to be one-off, perhaps cataclysmic events, two sources are now known to repeat and thus clearly have a longer-lived central engine. Beyond understanding how they are created, there is also the prospect of using FRBs -as with pulsars -to probe the extremes of the Universe as well as the otherwise invisible intervening medium. Such studies will be aided by the high implied all-sky event rate: there is a detectable FRB roughly once every minute occurring somewhere on the sky. The fact that less than a hundred FRB sources have been discovered in the last decade is largely due to the small fields-of-view of current radio telescopes. A new generation of wide-field instruments is now coming online, however, and these will be capable of detecting multiple FRBs per day. We are thus on the brink of further breakthroughs in the short-duration radio transient phase space, which will be critical for differentiating between the many proposed theories for the origin of FRBs. In this review, we give an observational and theoretical introduction at a level that is accessible to astronomers entering the field.

show abstract

Section: Candidate Groupingmentioning

confidence: 99%

Fast radio bursts

Petroff

Hessels

Lorimer

2019

Astron Astrophys Rev

622

471

View full text Add to dashboard Cite

show abstract

“…ANNs (Bethapudi & Desai, ; Bilicki et al, ; Ciuca & Hernández, ; Fujimoto, Fukushima, & Murase, ; Ho, ; Marchetti et al, ), RF methods (Goulding et al, ; Hedges et al, ; Nadler et al, ; Pang et al, ; Reis et al, ; Schindler et al, ; Tachibana & Miller, ), and SVM algorithms (Hartley et al, ; Hui et al, ; Kong et al, ; Yan et al, ; Zhang, Zhang, & Zhao, ) have been used extensively across most data types. CNNs are more suitable for image‐style data (see above), although they have been used successfully with one‐dimensional light curves (Shallue & Vanderburg, , identification and ranking of transiting exoplanet candidates in Kepler light curves, including the discovery of two new exoplanets) and time series (George & Huerta, , , identification of gravitational wave signatures within noisy time series data—a solution that scales better than template matching as the number of templates grows).…”

Section: Machine Learning and Artificial Intelligence In Astronomymentioning

confidence: 99%

Surveying the reach and maturity of machine learning and artificial intelligence in astronomy

Fluke

Jacobs

2019

WIREs Data Min & Knowl

113

View full text Add to dashboard Cite

Machine learning (automated processes that learn by example in order to classify, predict, discover, or generate new data) and artificial intelligence (methods by which a computer makes decisions or discoveries that would usually require human intelligence) are now firmly established in astronomy. Every week, new applications of machine learning and artificial intelligence are added to a growing corpus of work. Random forests, support vector machines, and neural networks are now having a genuine impact for applications as diverse as discovering extrasolar planets, transient objects, quasars, and gravitationally lensed systems, forecasting solar activity, and distinguishing between signals and instrumental effects in gravitational wave astronomy. This review surveys contemporary, published literature on machine learning and artificial intelligence in astronomy and astrophysics. Applications span seven main categories of activity: classification, regression, clustering, forecasting, generation, discovery, and the development of new scientific insights. These categories form the basis of a hierarchy of maturity, as the use of machine learning and artificial intelligence emerges, progresses, or becomes established. This article is categorized under: Application Areas > Science and Technology Fundamental Concepts of Data and Knowledge > Motivation and Emergence of Data Mining Technologies > Machine Learning

show abstract

“…Many recent works have applied data analytics to classification problems from radio astronomy (see [10] and [24] and references). While execution performance for scientific applications with Big Data has been considered in other fields (see [1], [2], [8], [25]) and other areas of astronomy (see [5], [11], [13], [22]), most of the literature from pulsar astronomy, including our previous works [10] and [24], are focused on classification performance with little consideration for execution performance.…”

Section: Related Workmentioning

confidence: 99%

“…In stage one, the raw data are pre-processed into a series of SPE files. In stage two, the SPE files are fed into a customized DBSCAN clustering algorithm [24] which solves problems specific to radio astronomy clustering, such as the merging of clusters from one single pulse that appear disparate due to artifacts of data processing. After identifying clusters of associated SPEs in the DM vs Time space, we extract characteristic features from them to create a series of files describing the clusters found.…”

Section: Scalable Single Pulse Identification and Classificationmentioning

confidence: 99%

“…As data sets become very large, candidate identification is increasingly processor intensive and becomes an execution performance bottleneck in the process. Our previous work focused on the identification and classification of dispersed pulse groups (DPGs) [10], and developing clustering algorithms to identify groups of single pulse events [24] in candidate plots generated by running PRESTO's single pulse processing tool, sin le_pulse_search.p .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Solutions for Automated Single Pulse Identification and Classification in Radio Astronomy

Devine

Goševa-Popstojanova

Pang

2018

Proceedings of the 47th International Conference on Parallel Processing

Self Cite

View full text Add to dashboard Cite

Data collection for scientific applications is increasing exponentially and is forecasted to soon reach peta-and exabyte scales. Applications which process and analyze scientific data must be scalable and focus on execution performance to keep pace. In the field of radio astronomy, in addition to increasingly large datasets, tasks such as the identification of transient radio signals from extrasolar sources are computationally expensive. We present a scalable approach to radio pulsar detection written in Scala that parallelizes candidate identification to take advantage of in-memory task processing using Apache Spark on a YARN distributed system. Furthermore, we introduce a novel automated multiclass supervised machine learning technique that we combine with feature selection to reduce the time required for candidate classification. Experimental testing on a Beowulf cluster with 15 data nodes shows that the parallel implementation of the identification algorithm offers a speedup of up to 5X that of a similar multithreaded implementation. Further, we show that the combination of automated multiclass classification and feature selection speeds up the execution performance of the RandomForest machine learning algorithm by an average of 54% with less than a 2% average reduction in the algorithm's ability to correctly classify pulsars. The generalizability of these results is demonstrated by using two real-world radio astronomy data sets.

show abstract

A novel single-pulse search approach to detection of dispersed radio pulses using clustering and supervised machine learning

Cited by 26 publications

References 22 publications

Fast radio bursts

Fast radio bursts

Surveying the reach and maturity of machine learning and artificial intelligence in astronomy

Scalable Solutions for Automated Single Pulse Identification and Classification in Radio Astronomy

Contact Info

Product

Resources

About