It has been widely observed that large neural networks can be pruned to a small fraction of their original size, with little loss in accuracy, by typically following a time-consuming "train, prune, re-train" approach. Frankle & Carbin (2018) conjecture that we can avoid this by training lottery tickets, i.e., special sparse subnetworks found at initialization, that can be trained to high accuracy. However, a subsequent line of work presents concrete evidence that current algorithms for finding trainable networks at initialization, fail simple baseline comparisons, e.g., against training random sparse subnetworks. Finding lottery tickets that train to better accuracy compared to simple baselines remains an open problem. In this work, we partially resolve this open problem by discovering rare gems: subnetworks at initialization that attain considerable accuracy, even before training. Refining these rare gems-by means of finetuning-beats current baselines and leads to accuracy competitive or better than magnitude pruning methods.

Multivariate autoregressive (MVAR) model estimation enables assessment of causal interactions in brain networks. However, accurately estimating MVAR models for high-dimensional electrophysiological recordings is challenging due to the extensive data requirements. Hence, the applicability of MVAR models for study of brain behavior over hundreds of recording sites has been very limited. Prior work has focused on different strategies for selecting a subset of important MVAR coefficients in the model and is motivated by the potential of MVAR models and the data requirements of conventional least-squares estimation algorithms. Here we propose incorporating prior information, such as fMRI, into MVAR model estimation using a weighted group LASSO regularization strategy. The proposed approach is shown to reduce data requirements by a factor of two relative to the recently proposed group LASSO method of Endemann et al. (2022) while resulting in models that are both more parsimonious and have higher fidelity to the ground truth. The effectiveness of the method is demonstrated using simulation studies of physiologically realistic MVAR models derived from iEEG data. The robustness of the approach to deviations between the conditions under which the prior information and iEEG data is obtained is illustrated using models from data collected in different sleep stages. This approach will allow accurate effective connectivity analyses over short time scales, facilitating investigations of causal interactions in the brain underlying perception and cognition during rapid transitions in behavioral state.

No abstract