No abstract

Understanding the learning dynamics and inductive bias of neural networks (NNs) is hindered by the opacity of the relationship between NN parameters and the function represented. Partially, this is due to symmetries inherent within the NN parameterization, allowing multiple different parameter settings to result in an identical output function, resulting in both an unclear relationship and redundant degrees of freedom. The NN parameterization is invariant under two symmetries: permutation of the neurons and a continuous family of transformations of the scale of weight and bias parameters. We propose taking a quotient with respect to the second symmetry group and reparametrizing ReLU NNs as continuous piecewise linear splines. Using this spline lens, we study learning dynamics in shallow univariate ReLU NNs, finding unexpected insights and explanations for several perplexing phenomena. We develop a surprisingly simple and transparent view of the structure of the loss surface, including its critical and fixed points, Hessian, and Hessian spectrum. We also show that standard weight initializations yield very flat initial functions, and that this flatness, together with overparametrization and the initial weight scale, is responsible for the strength and type of implicit regularization, consistent with previous work. Our implicit regularization results are complementary to recent work, showing that initialization scale critically controls implicit regularization via a kernel-based argument. Overall, removing the weight scale symmetry enables us to prove these results more simply and enables us to prove new results and gain new insights while offering a far more transparent and intuitive picture. Looking forward, our quotiented spline-based approach will extend naturally to the multivariate and deep settings, and alongside the kernel-based view, we believe it will play a foundational role in efforts to understand neural networks. Videos of learning dynamics using a spline-based visualization are available at http://shorturl.at/tFWZ2.

No abstract

No abstract

Intravenous (IV) fluid regulation is necessary in developing nations to prevent IV-overhydration in the pediatric patients of low-resource hospitals. Traditionally, regulation is achieved by calculating the total fluid outflow from an IV bag and restricting flow before the patient is injected with dangerous levels of fluid. However, standard fluid regulation devices include infusion pumps and burettes, which are costly and often ill-suited for low-resource environments. This research proposes a low-cost, easy to use device that regulates the volume of intravenous (IV) fluid delivered to a patient in a low-resource clinical setting. Laboratory accuracy tests (N = 32) over a range of clinically-relevant fluid volumes yielded a median and max error of 4 and 8mL respectively, falling within specific error thresholds (p < le-5). Non-clinical usability tests (N = 25) showed median and max device setup times to be 40 and 55 seconds respectively (p < le-5). Additionally, all participants found the device “easy to use” and were able to set up and use the device with less than 20 minutes of training.