Stem cell therapies have enormous potential for treating many debilitating diseases, including heart failure, stroke and traumatic brain injury. For maximal efficacy, these therapies require targeted cell delivery to specific tissues followed by successful cell engraftment. However, targeted delivery remains an open challenge. As one example, it is common for intravenous deliveries of mesenchymal stem cells (MSCs) to become entrapped in lung microvasculature instead of the target tissue. Hence, a robust, quantitative imaging method would be essential for developing efficacious cell therapies. Here we show that Magnetic Particle Imaging (MPI), a novel technique that directly images iron-oxide nanoparticle-tagged cells, can longitudinally monitor and quantify MSC administration in vivo. MPI offers near-ideal image contrast, depth penetration, and robustness; these properties make MPI both ultra-sensitive and linearly quantitative. Here, we imaged, for the first time, the dynamic trafficking of intravenous MSC administrations using MPI. Our results indicate that labeled MSC injections are immediately entrapped in lung tissue and then clear to the liver within one day, whereas standard iron oxide particle (Resovist) injections are immediately taken up by liver and spleen. Longitudinal MPI-CT imaging also indicated a clearance half-life of MSC iron oxide labels in the liver at 4.6 days. Finally, our ex vivo MPI biodistribution measurements of iron in liver, spleen, heart, and lungs after injection showed excellent agreement (R2 = 0.943) with measurements from induction coupled plasma spectrometry. These results demonstrate that MPI offers strong utility for noninvasively imaging and quantifying the systemic distribution of cell therapies and other therapeutic agents.

State-of-the-art natural language understanding classification models follow twostages: pre-training a large language model on an auxiliary task, and then finetuning the model on a task-specific labeled dataset using cross-entropy loss. Crossentropy loss has several shortcomings that can lead to sub-optimal generalization and instability. Driven by the intuition that good generalization requires capturing the similarity between examples in one class and contrasting them with examples in other classes, we propose a supervised contrastive learning (SCL) objective for the fine-tuning stage. Combined with cross-entropy, the SCL loss we propose obtains improvements over a strong RoBERTa-Large baseline on multiple datasets of the GLUE benchmark in both the high-data and low-data regimes, and it does not require any specialized architecture, data augmentation of any kind, memory banks, or additional unsupervised data. We also demonstrate that the new objective leads to models that are more robust to different levels of noise in the training data, and can generalize better to related tasks with limited labeled task data.

Unsupervised pre-training has led to much recent progress in natural language understanding. In this paper, we study self-training as another way to leverage unlabeled data through semi-supervised learning. To obtain additional data for a specific task, we introduce SentAugment, a data augmentation method which computes task-specific query embeddings from labeled data to retrieve sentences from a bank of billions of unlabeled sentences crawled from the web. Unlike previous semisupervised methods, our approach does not require in-domain unlabeled data and is therefore more generally applicable. Experiments show that self-training is complementary to strong RoBERTa baselines on a variety of tasks. Our augmentation approach leads to scalable and effective self-training with improvements of up to 2.6% on standard text classification benchmarks. Finally, we also show strong gains on knowledge-distillation and few-shot learning.

Inductive sensor-based measurement techniques are useful for a wide range of biomedical applications. However, optimizing the noise performance of these sensors is challenging at broadband frequencies, owing to the frequency-dependent reactance of the sensor. In this work, we describe the fundamental limits of noise performance and bandwidth for these sensors in combination with a low-noise amplifier. We also present three equivalent methods of noise matching to inductive sensors using transformer-like network topologies. Finally, we apply these techniques to improve the noise performance in magnetic particle imaging, a new molecular imaging modality with excellent detection sensitivity. Using a custom noise-matched amplifier, we experimentally demonstrate an 11-fold improvement in noise performance in a small animal magnetic particle imaging scanner.

Unsupervised pre-training has led to much recent progress in natural language understanding. In this paper, we study self-training as another way to leverage unlabeled data through semi-supervised learning. To obtain additional data for a specific task, we introduce SentAugment, a data augmentation method which computes task-specific query embeddings from labeled data to retrieve sentences from a bank of billions of unlabeled sentences crawled from the web. Unlike previous semisupervised methods, our approach does not require in-domain unlabeled data and is therefore more generally applicable. Experiments show that self-training is complementary to strong RoBERTa baselines on a variety of tasks. Our augmentation approach leads to scalable and effective self-training with improvements of up to 2.6% on standard text classification benchmarks. Finally, we also show strong gains on knowledge-distillation and few-shot learning. * Equal contribution.An alternative semi-supervised technique is pretraining (Dai and Le, 2015;Radford et al., 2018;Howard and Ruder, 2018;Devlin et al., 2018), which has led to large improvements for natural language understanding compared to purely supervised learning. In that case, models are first trained on an auxiliary task, such as language modeling, followed by fine-tuning on the task of interest.A natural question is the following: do pretraining and self-training capture the same information, or are they complementary? Recently, Zoph et al. ( 2020) studied this question in the context of image recognition, showing that selftraining was helpful, even in addition to pretraining. However, their study mostly considers supervised pre-training, in which models were trained on ImageNet classification. Moreover, in cases where large amounts of supervised data were available for the downstream task, pre-training was not helpful, even without self-training. This is in contrast to natural language understanding for which language modeling pre-training is a very strong baseline that leads to large improvements for all the tasks we consider.An important ingredient for self-training, and semi-supervised learning in general, is the unannotated data and the fact that it comes from the same domain as the downstream task. Existing work, such as UDA (Xie et al., 2019), selftraining (He et al., 2019Xie et al., 2020) and back-translation for machine translation (Bojar and Tamchyna, 2011;Sennrich et al., 2015;Edunov et al., 2018), assumes the existence of unannotated data in the same domain as the downstream task. This assumption limits the broad application of such semi-supervised methods, in particular in the case of low-resource downstream tasks. A second important question is thus: how can we obtain large amounts of unannotated data from specific domains?