SpikeForest: reproducible web-facing ground-truth validation of automated neural spike sorters

Einevoll

2019

Preprint

Self Cite

When recording neural activity from extracellular electrodes, both in vivo and in vitro, spike sorting is a required and very important processing step that allows for identification of single neurons' activity. Spike sorting is a complex algorithmic procedure, and in recent years many groups have attempted to tackle this problem, resulting in numerous methods and software packages. However, validation of spike sorting techniques is complicated. It is an inherently unsupervised problem and it is hard to find universal metrics to evaluate performance. Simultaneous recordings that combine extracellular and patch-clamp or juxtacellular techniques can provide ground-truth data to evaluate spike sorting methods. However, their utility is limited by the fact that only a few cells can be measured at the same time. Simulated ground-truth recordings can provide a powerful alternative mean to rank the performance of spike sorters. We present here MEArec, a Pythonbased software which permits flexible and fast simulation of extracellular recordings. MEArec allows users to generate extracellular signals on various customizable electrode designs and can replicate various problematic aspects for spike sorting, such as bursting, spatio-temporal overlapping events, and drifts. We expect MEArec will provide a common testbench for spike sorting development and evaluation, in which spike sorting developers can rapidly generate and evaluate the performance of their algorithms.

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

MEArec: a fast and customizable testbench simulator for ground-truth extracellular spiking activity

Einevoll

2019

Preprint

Self Cite

“…Despite the development and widespread use of automatic spike sorters, there still exist no clear standards for how spike sorting should be performed or evaluated [62,11,18,47]. Research labs that are beginning to experiment with high-density extracellular recordings have to choose from a multitude of spike sorters, data processing algorithms, file formats, and curation tools just to analyze their first recording.…”

Section: Introductionmentioning

confidence: 99%

“…Research labs that are beginning to experiment with high-density extracellular recordings have to choose from a multitude of spike sorters, data processing algorithms, file formats, and curation tools just to analyze their first recording. As trying out multiple spike sorting pipelines is time-consuming and technically challenging, many labs choose one and stick to it as their de facto solution [47]. This has led to a fragmented software ecosystem which challenges reproducibility, benchmarking, and collaboration among different research labs.…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, these tools do little to improve the evaluation and comparison of spike sorting performance which is still a relatively unsolved problem in electrophysiology. An exception to this is SpikeForest [47], a recently developed open-source software suite that benchmarks 10 automated spike sorting algorithms against an extensive database of ground truth recordings 1 . Despite these developments, there exists a need for an up-to-date spike sorting framework that can standardize the usage and evaluation of modern algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SpikeInterface, a unified framework for spike sorting

Hurwitz

Magland

et al. 2019

Preprint

Self Cite

Given the importance of understanding single-neuron activity, much development has been directed towards improving the performance and automation of spike sorting. These developments, however, introduce new challenges, such as file format incompatibility and reduced interoperability, that hinder benchmarking and preclude reproducible analysis. To address these limitations, we developed SpikeInterface, a Python framework designed to unify preexisting spike sorting technologies into a single codebase and to standardize extracellular data file operations. With a few lines of code and regardless of the underlying data format, researchers can: run, compare, and benchmark most modern spike sorting algorithms; pre-process, post-process, and visualize extracellular datasets; validate, curate, and export sorting outputs; and more. In this paper, we provide an overview of SpikeInterface and, with applications to both real and simulated extracellular datasets, demonstrate how it can improve the accessibility, reliability, and reproducibility of spike sorting in preparation for the widespread use of large-scale electrophysiology. * cole.hurwitz@ed.ac.uk 1 We utilize Python as it is open-source, free, and increasingly popular in the neuroscience community [50,29]. 2 https://github.com/SpikeInterface 2

MEArec: A Fast and Customizable Testbench Simulator for Ground-truth Extracellular Spiking Activity

Einevoll

2020

Neuroinform

When recording neural activity from extracellular electrodes, both in vivo and in vitro, spike sorting is a required and very important processing step that allows for identification of single neurons’ activity. Spike sorting is a complex algorithmic procedure, and in recent years many groups have attempted to tackle this problem, resulting in numerous methods and software packages. However, validation of spike sorting techniques is complicated. It is an inherently unsupervised problem and it is hard to find universal metrics to evaluate performance. Simultaneous recordings that combine extracellular and patch-clamp or juxtacellular techniques can provide ground-truth data to evaluate spike sorting methods. However, their utility is limited by the fact that only a few cells can be measured at the same time. Simulated ground-truth recordings can provide a powerful alternative mean to rank the performance of spike sorters. We present here , a Python-based software which permits flexible and fast simulation of extracellular recordings. allows users to generate extracellular signals on various customizable electrode designs and can replicate various problematic aspects for spike sorting, such as bursting, spatio-temporal overlapping events, and drifts. We expect will provide a common testbench for spike sorting development and evaluation, in which spike sorting developers can rapidly generate and evaluate the performance of their algorithms.