Large-scale pre-trained language models such as BERT have brought significant improvements to NLP applications. However, they are also notorious for being slow in inference, which makes them difficult to deploy in realtime applications. We propose a simple but effective method, DeeBERT, to accelerate BERT inference. Our approach allows samples to exit earlier without passing through the entire model. Experiments show that DeeBERT is able to save up to ∼40% inference time with minimal degradation in model quality. Further analyses show different behaviors in the BERT transformer layers and also reveal their redundancy. Our work provides new ideas to efficiently apply deep transformer-based models to downstream tasks. Code is available at https://github.com/castorini/ DeeBERT.

Since the working mechanism of triboelectric nanogenerators (TENGs) is based on triboelectrification and electrostatic induction, it is necessary to understand the effects of the inherent properties of dielectric materials on the performance of TENGs. In this study, the relationship between the relative permittivity and the performance of TENGs was demonstrated by fabricating TENGs using both pure oxide materials (SiO 2 , Al 2 O 3 , HfO 2 , Ta 2 O 5 and TiO 2 ) and oxide-PMMA composites. As oxide materials and PMMA are triboelectrically positive, PTFE film was selected as the counter tribo-material, which has highly negative triboelectric polarity. The triboelectric series of the above-mentioned oxides was experimentally organized to clarify the major parameter for the performance of TENGs. The electrical data values for both oxides and composites clearly showed a tendency to increase as the relative permittivity of the tribo-material increased. It is also well-matched with the theoretical analysis between the electrical performances (e.g. open-circuit voltage) and relative permittivity. However, such a tendency is not observed with the triboelectric polarity. Due to the tribo-material's high relative permittivity, an opencircuit voltage of 124.1 V, a short-circuit current of 14.88 mA and a power of 392.08 mW were obtained in a pure TiO 2 thin film.

BackgroundFibromyalgia (FM) is a disorder characterized by chronic widespread pain and frequently associated with other symptoms. Patients with FM commonly report cognitive complaints, including memory problem. The objective of this study was to investigate the differences in neural correlates of working memory between FM patients and healthy subjects, using functional magnetic resonance imaging (MRI).Methodology/Principal FindingsNineteen FM patients and 22 healthy subjects performed an n-back memory task during MRI scan. Functional MRI data were analyzed using within- and between-group analysis. Both activated and deactivated brain regions during n-back task were evaluated. In addition, to investigate the possible effect of depression and anxiety, group analysis was also performed with depression and anxiety level in terms of Beck depression inventory (BDI) and Beck anxiety inventory (BAI) as a covariate. Between-group analyses, after controlling for depression and anxiety level, revealed that within the working memory network, inferior parietal cortex was strongly associated with the mild (r = 0.309, P = 0.049) and moderate (r = 0.331, P = 0.034) pain ratings. In addition, between-group comparison revealed that within the working memory network, the left DLPFC, right VLPFC, and right inferior parietal cortex were associated with the rating of depression and anxiety?Conclusions/SignificanceOur results suggest that the working memory deficit found in FM patients may be attributable to differences in neural activation of the frontoparietal memory network and may result from both pain itself and depression and anxiety associated with pain.

An aqueous-based approach for the scalable synthesis of nitrogen-doped porous carbons with high specific surface area (SSA) and high nitrogen content is presented. Low molecular weight polyacrylonitrile (PAN) is solubilized in water in the presence of ZnCl that also acts as a volatile porogen during PAN pyrolysis to form mesoporous structures with significantly increased SSA. By templating with commercial SiO nanoparticles, nanocellulose fillers or filter paper, nanocarbons with SSA = 1776, 1366, and 1501 m/g, respectively and 10 wt % N content were prepared. The materials formed by this benign process showed excellent catalytic activity in oxygen reduction reaction via the four-electron mechanism.

Methamphetamine (MA) abusers commonly exhibit socially problematic behaviors, such as diminished empathy, decreased emotional regulation and interpersonal violence, which may be attributable to alterations in emotional experience. However, few studies have used functional MRI to examine directly the emotional experience of threatening or fearful non-face images in MA abusers. In this study, we investigated possible differences in neural correlates of negative emotional experiences between abstinent MA abusers and healthy subjects using complex visual scenes depicting fear or threat derived from the International Affective Picture System. In within-group analyses, healthy subjects and MA abusers activated a similarly distributed cortical network, prominently including the amygdala, fusiform gyrus, parahippocampal gyrus, ventrolateral prefrontal cortex and inferior frontal cortex. In between-group analyses, however, MA abusers showed a reduced activation in the bilateral dorsolateral prefrontal cortex and insula, and increased activation in the fusiform gyrus, hippocampus, parahippocampal gyrus and posterior cingulate cortex, relative to healthy subjects. Hypoactivation of the insula in MA abusers relative to healthy subjects suggests that the ability to have an emotional response to threatening scenes and empathy for another's pain could be compromised in MA abusers. Hyperactivity in the fusiform gyrus, parahippocampal gyrus and posterior cingulate cortex in MA abusers relative to healthy subjects indicates that threatening and fearful images from the International Affective Picture System may remind MA abusers of episodic memory related to similar experiences. Therefore, functional impairment of these neural networks in MA abusers may contribute to altered emotional experience in social interactions, which could lead to increased negative mood and stress in interpersonal communication.