Neuroscience is experiencing a data revolution in which simultaneous recording of many hundreds or thousands of neurons is revealing structure in population activity that is not apparent from single-neuron responses. This structure is typically extracted from trial-averaged data. Single-trial analyses are challenging due to incomplete sampling of the neural population, trial-to-trial variability, and fluctuations in action potential timing. Here we introduce Latent Factor Analysis via Dynamical Systems (LFADS), a deep learning method to infer latent dynamics from single-trial neural spiking data. LFADS uses a nonlinear dynamical system (a recurrent neural network) to infer the dynamics underlying observed population activity and to extract 'de-noised' single-trial firing rates from neural spiking data. We apply LFADS to a variety of monkey and human motor cortical datasets, demonstrating its ability to predict observed behavioral variables with unprecedented accuracy, extract precise estimates of neural dynamics on single trials, infer perturbations to those dynamics that correlate with behavioral choices, and combine data from non-overlapping recording sessions (spanning months) to improve inference of underlying dynamics. In summary, LFADS leverages all observations of a neural population's activity to accurately model its dynamics on single trials, opening the door to a detailed understanding of the role of dynamics in performing computation and ultimately driving behavior.Increasing evidence suggests that in many brain areas, such as the motor and prefrontal cortices, the activity of large populations of neurons, termed the neural population state, is often well-described by low-dimensional dynamics [e.g. (Afshar et al. 2011; Harvey, Coen, and Tank 2012; Kaufman et al. 2014;Sadtler et al. 2014;Kobak et al. 2016a) ]. Recovering these dynamics on single trials is essential for illuminating the relationship between neural population activity and behavior, and for advancing therapeutic neurotechnologies such as closed-loop deep brain stimulation and brain-machine interfaces. However, recovering population dynamics All rights reserved. No reuse allowed without permission.(which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint . http://dx.doi.org/10.1101/152884 doi: bioRxiv preprint first posted online Jun. 20, 2017; on single trials is difficult due to trial-to-trial variability (e.g. in behavior or arousal) and fluctuations in the spiking of individual neurons. Even with dramatic increases in the numbers of neurons that can be simultaneously recorded using multichannel electrode arrays or optical imaging, accurately recovering population dynamics from single trials remains a significant challenge for data-analysis methods.Standard analyses sacrifice single-trial information for the sake of better estimates of trial-averaged neural states (Ahrens et al. 2012; Kobak et al. 2016b) . Techniques for extrac...