Pretrained language models such as BERT, GPT have shown great effectiveness in language understanding. The auxiliary predictive tasks in existing pretraining approaches are mostly defined on tokens, thus may not be able to capture sentence-level semantics very well. To address this issue, we propose CERT: Contrastive self-supervised Encoder Representations from Transformers, which pretrains language representation models using contrastive selfsupervised learning at the sentence level. CERT creates augmentations of original sentences using back-translation. Then it finetunes a pretrained language encoder (e.g., BERT) by predicting whether two augmented sentences originate from the same sentence. CERT is simple to use and can be flexibly plugged into any pretraining-finetuning NLP pipeline. We evaluate CERT on 11 natural language understanding tasks in the GLUE benchmark where CERT outperforms BERT on 7 tasks, achieves the same performance as BERT on 2 tasks, and performs worse than BERT on 2 tasks. On the averaged score of the 11 tasks, CERT outperforms BERT. The data and code are available at https://github.com/UCSD-AI4H/ CERT. †The work was done during internship at UCSD. . * Equal contribution.

Background: Accumulating evidence has suggested that there is a positive association between asthma and cardiovascular diseases (CVDs), implying a common architecture between them. However, the shared genetic architecture and causality of asthma and CVDs remain unclear.Methods: Based on the genome-wide association study (GWAS) summary statistics of recently published studies, our study examined the genetic correlation, shared genetic variants, and causal relationship between asthma (N = 127,669) and CVDs (N = 86,995–521,612). Statistical methods included high-definition likelihood (HDL), cross-trait meta-analyses of large-scale GWAS, transcriptome-wide association studies (TWAS), and Mendelian randomization (MR).Results: First, we observed a significant genetic correlation between asthma and heart failure (HF) (Rg = 0.278, P = 5 × 10−4). Through cross-trait analyses, we identified a total of 145 shared loci between asthma and HF. Fifteen novel loci were not previously reported for association with either asthma or HF. Second, we mapped these 145 loci to a total of 99 genes whose expressions are enriched in a broad spectrum of tissues, including the seminal vesicle, tonsil, appendix, spleen, skin, lymph nodes, breast, cervix and uterus, skeletal muscle, small intestine, lung, prostate, cardiac muscle, and liver. TWAS analysis identified five significant genes shared between asthma and HF in tissues from the hemic and immune system, digestive system, integumentary system, and nervous system. GSDMA, GSDMB, and ORMDL3 are statistically independent genetic effects from all shared TWAS genes between asthma and HF. Third, through MR analysis, genetic liability to asthma was significantly associated with heart failure at the Bonferroni-corrected significance level. The odds ratio (OR) is 1.07 [95% confidence interval (CI): 1.03–1.12; p = 1.31 × 10−3] per one-unit increase in loge odds of asthma.Conclusion: These findings provide strong evidence of genetic correlations and causal relationship between asthma and HF, suggesting a shared genetic architecture for these two diseases.

The role of fatty acid-binding proteins (FABPs) in atherosclerosis has been investigated. The aim of this study was to verify the hypothesis that higher levels of serum fatty acid-binding protein 4 (FABP4) could be a prognostic factor in Chinese patients with type 2 diabetes (T2DM) and acute ischemic stroke (AIS). From September 2015 to August 2016, consecutive first-ever AIS patients combined with T2DM were included in this study. FABP4, NIH stroke scale (NIHSS), and conventional risk factors were evaluated to determine their value to predict functional outcomes within 3 months. Multivariate analyses were performed using logistic regression models. We measured FABP4 in 329 patients. The median age of patients included in this study was 63 (IQR, 56-72) years and 45.9% were women. FABP4 serum levels were obtained at a median of 8.5 h (IQR, 4.0-14.0 h) after the stroke onset with a median value of 21.4 ng/ml (IQR, 15.6-28.2 ng/ml). In multivariable models, FABP4 remained an independent stroke severity predictor with an adjusted OR of 1.05 (95% CI, 1.02-1.09). In multivariate models comparing the third (odd ratio (OR), 2.25; 95% confidence interval (CI), 1.59-3.54) and fourth quartiles (OR, 3.75; 95% CI, 2.48-5.03) against the first quartile of the FABP4, levels of FABP4 were associated with poor functional outcome. At 3 months, 38 patients (11.6%; 95%CI, 8.1-15.0%) had died. The mortality distribution across the FABP4 quartiles ranged between 3.7% (first quartile) and 20.7% (fourth quartile). Elevation of FABP4 is associated with an increased risk of death and poor functional outcome events in patients with type 2 diabetes and acute ischemic stroke and is independent of other established clinical risk predictors and biomarkers.

No abstract

No abstract