Raising the Profile of Research Software

Akhmerov, Anton; Cruz, Maria; Drost, Niels; Hof, Cees H.J.; Knapen, Tomas; Kuzak, Mateusz; Martínez-Ortiz, Carlos; Velden, Yasemin Turkyilmaz-van der; Werkhoven, Ben van

doi:10.5281/zenodo.3378572

Cited by 5 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Executable model descriptions are, in part, already expressed in the simulator independent language PyNN (Davison et al, 2009 ), but there is no support by a common benchmarking framework, and a focus is set on correctness rather than computational efficiency. The emerging field of Research Software Engineering (RSE) is studying how, in the scientific setting, reliable and sustainable software can be developed, developers can be educated for this purpose, and science organizations and politics can be made aware of its strategic relevance (Manifesto 25 and Akhmerov et al, 2019 ). Obvious differences to software engineering in the industrial setting include research code being developed by scientists rather than experienced software developers, the time-constrained and thesis-bound nature of scientific projects, and the continuous integration of new contributors into the development process.…”

Section: Discussionmentioning

confidence: 99%

A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations

Albers

Pronold

Kurth

et al. 2022

Front. Neuroinform.

View full text Add to dashboard Cite

Modern computational neuroscience strives to develop complex network models to explain dynamics and function of brains in health and disease. This process goes hand in hand with advancements in the theory of neuronal networks and increasing availability of detailed anatomical data on brain connectivity. Large-scale models that study interactions between multiple brain areas with intricate connectivity and investigate phenomena on long time scales such as system-level learning require progress in simulation speed. The corresponding development of state-of-the-art simulation engines relies on information provided by benchmark simulations which assess the time-to-solution for scientifically relevant, complementary network models using various combinations of hardware and software revisions. However, maintaining comparability of benchmark results is difficult due to a lack of standardized specifications for measuring the scaling performance of simulators on high-performance computing (HPC) systems. Motivated by the challenging complexity of benchmarking, we define a generic workflow that decomposes the endeavor into unique segments consisting of separate modules. As a reference implementation for the conceptual workflow, we develop beNNch: an open-source software framework for the configuration, execution, and analysis of benchmarks for neuronal network simulations. The framework records benchmarking data and metadata in a unified way to foster reproducibility. For illustration, we measure the performance of various versions of the NEST simulator across network models with different levels of complexity on a contemporary HPC system, demonstrating how performance bottlenecks can be identified, ultimately guiding the development toward more efficient simulation technology.

show abstract

Section: Discussionmentioning

confidence: 99%

A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations

Albers

Pronold

Kurth

et al. 2022

Front. Neuroinform.

View full text Add to dashboard Cite

show abstract

“…Executable model descriptions are, in part, already expressed in the simulator independent language PyNN (Davison et al, 2009), but there is no support by a common benchmarking framework, and a focus is set on correctness rather than computational efficiency. The emerging field of Research Software Engineering (RSE) is studying how, in the scientific setting, reliable and sustainable software can be developed, developers can be educated for this purpose, and science organizations and politics can be made aware of its strategic relevance (Manifesto 23 and Akhmerov et al, 2019). Obvious differences to software engineering in the industrial setting include research code being developed by scientists rather than experienced software developers, the time-constrained and thesis-bound nature of scientific projects, and the continuous integration of new contributors into the development process.…”

Section: Discussionmentioning

confidence: 99%

A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations

Albers,

Pronold,

Kurth

et al. 2021

Preprint

View full text Add to dashboard Cite

Modern computational neuroscience strives to develop complex network models to explain dynamics and function of brains in health and disease. This process goes hand in hand with advancements in the theory of neuronal networks and increasing availability of detailed anatomical data on brain connectivity. Large-scale models that study interactions between multiple brain areas with intricate connectivity and investigate phenomena on long time scales such as systemlevel learning require progress in simulation speed. The corresponding development of stateof-the-art simulation engines relies on information provided by benchmark simulations which assess the time-to-solution for scientifically relevant, complementary network models using various combinations of hardware and software revisions. However, maintaining comparability of benchmark results is difficult due to a lack of standardized specifications for measuring the scaling performance of simulators on high-performance computing (HPC) systems. Motivated by the challenging complexity of benchmarking, we define a generic workflow that decomposes the endeavor into unique segments consisting of separate modules. As a reference implementation for the conceptual workflow, we develop beNNch: an open-source software framework for the configuration, execution, and analysis of benchmarks for neuronal network simulations. The framework records benchmarking data and metadata in a unified way to foster reproducibility. For illustration, we measure the performance of various versions of the NEST simulator across 1

show abstract

“…Therefore the competence in research software engineering and the acceptance of software development as an integral part of scientific methodology may have been more widespread at this time. There is a long-standing awareness that software development in science, including computational neuroscience [104,105], is subject to special conditions: most scientists are not trained programmers [106], and it is often difficult to receive proper credit for the time invested in developing software [107]. As a result such tasks regularly are assigned low priority and progress is slow.…”

Section: Historical Contextmentioning

confidence: 99%

Connectivity Concepts in Neuronal Network Modeling

Senk,

Kriener,

Djurfeldt

et al. 2021

Preprint

View full text Add to dashboard Cite

Sustainable research on computational models of neuronal networks requires published models to be understandable, reproducible, and extendable. Missing details or ambiguities about mathematical concepts and assumptions, algorithmic implementations, or parameterizations hinder progress. Such flaws are unfortunately frequent and one reason is a lack of readily applicable standards and tools for model description. Our work aims to advance complete and concise descriptions of network connectivity but also to guide the implementation of connection routines in simulation software and neuromorphic hardware systems. We first review models made available by the computational neuroscience community in the repositories ModelDB and Open Source Brain, and investigate the corresponding connectivity structures and their descriptions in both manuscript and code. The review comprises the connectivity of networks with diverse levels of neuroanatomical detail and exposes how connectivity is abstracted in existing description languages and simulator interfaces. We find that a substantial proportion of the published descriptions of connectivity is ambiguous. Based on this review, we derive a set of connectivity concepts for deterministically and probabilistically connected networks and also address networks embedded in metric space. Beside these mathematical and textual guidelines, we propose a unified graphical notation for network diagrams to facilitate an intuitive understanding of network properties. Examples of representative network models demonstrate the practical use of the ideas. We hope that the proposed standardizations will contribute to unambiguous descriptions and reproducible implementations of neuronal network connectivity in computational neuroscience.

show abstract

Raising the Profile of Research Software

Cited by 5 publications

References 0 publications

A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations

A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations

A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations

Connectivity Concepts in Neuronal Network Modeling

Contact Info

Product

Resources

About