Fastai: A Layered API for Deep Learning

Abstract

fastai is a deep learning library which provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains, and provides researchers with low-level components that can be mixed and matched to build new approaches. It aims to do both things without substantial compromises in ease of use, flexibility, or performance. This is possible thanks to a carefully layered architecture, which expresses common underlying patterns of many deep learning and data processing techniques in terms of decoupled abstractions. These abstractions can be expressed concisely and clearly by leveraging the dynamism of the underlying Python language and the flexibility of the PyTorch library. fastai includes: a new type dispatch system for Python along with a semantic type hierarchy for tensors; a GPU-optimized computer vision library which can be extended in pure Python; an optimizer which refactors out the common functionality of modern optimizers into two basic pieces, allowing optimization algorithms to be implemented in 4-5 lines of code; a novel 2-way callback system that can access any part of the data, model, or optimizer and change it at any point during training; a new data block API; and much more. We have used this library to successfully create a complete deep learning course, which we were able to write more quickly than using previous approaches, and the code was more clear. The library is already in wide use in research, industry, and teaching.

Cell segmentation using deep learning: comparing label and label-free approaches using hyper-labeled image stacks

“…To that end, we have prepared an annotated Jupyter notebook (see section II-F) detailing the process of dataset collection, ground-truth labeling and model training that requires minimal knowledge of deep learning to implement. For simplicity, many of the parameters used during training (hyperparameters) have been pre-set to reflect current best practices [18] while the remaining hyperparameters (learning rate, data augmentation parameters, class weights) may be optimized using submodules provided within the notebook. When collecting your hyper-labeled image stacks, the major requirement is that the subimage stacks used in training be representative of the expected experimental images.…”

“…However, the learning rate, number of training epochs, and cross entropy weights were determined experimentally. Learning rate was scheduled as a variation of the 1cycle policy [11], [18] ( Supplementary Fig. S2a,b).…”

“…The model used here was composed of a traditional ResNet34 [16] architecture for the descending arc path and a custom ascending arc that used pixel shuffling during upsampling to reduce checkerboard artifacts [17]. To make use of transfer learning, pretrained weights (provided by [18]) from the ResNet34 model were used for the descending arc while the ascending arc was randomly initialized.…”

“…The majority of the hyperparameters used represent bestpractice recommendations as described in [18]. However, the learning rate, number of training epochs, and cross entropy weights were determined experimentally.…”

USMPep: Universal Sequence Models for Major Histocompatibility Complex Binding Affinity Prediction

“…This work was supported by the Bundesministerium für Bildung und Forschung (BMBF) through the Berlin Big Data Center under Grant 01IS14013A and the Berlin Center for Machine Learning under Grant 01IS18037I. USMPep was implemented using PyTorch [24] and fast.ai [25].…”