Contextual word embedding models such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018) have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these models have been minimally explored on specialty corpora, such as clinical text; moreover, in the clinical domain, no publicly-available pre-trained BERT models yet exist. In this work, we address this need by exploring and releasing BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically. We demonstrate that using a domain-specific model yields performance improvements on three common clinical NLP tasks as compared to nonspecific embeddings. These domainspecific models are not as performant on two clinical de-identification tasks, and argue that this is a natural consequence of the differences between de-identified source text and synthetically non de-identified task text.

In vitro delivery of the diphtheria toxin catalytic (C) domain from the lumen of purified early endosomes to the external milieu requires the addition of both ATP and a cytosolic translocation factor (CTF) complex. Using the translocation of C-domain ADP-ribosyltransferase activity across the endosomal membrane as an assay, the CTF complex activity was 650–800-fold purified from human T cell and yeast extracts, respectively. The chaperonin heat shock protein (Hsp) 90 and thioredoxin reductase were identified by mass spectrometry sequencing in CTF complexes purified from both human T cell and yeast. Further analysis of the role played by these two proteins with specific inhibitors, both in the in vitro translocation assay and in intact cell toxicity assays, has demonstrated their essential role in the productive delivery of the C-domain from the lumen of early endosomes to the external milieu. These results confirm and extend earlier observations of diphtheria toxin C-domain unfolding and refolding that must occur before and after vesicle membrane translocation. In addition, results presented here demonstrate that thioredoxin reductase activity plays an essential role in the cytosolic release of the C-domain. Because analogous CTF complexes have been partially purified from mammalian and yeast cell extracts, results presented here suggest a common and fundamental mechanism for C-domain translocation across early endosomal membranes.

Although the structural gene for diphtheria toxin, tox, is carried by a family of closely related corynebacteriophages, the regulation of lox expression is controfle, to a large extent, by its bacterial host Corynebacleium diphtheriae. Optimal yields oftox gene products are obtained only when iron becomes the growth-rate-limiting substrate. Previous studies suggest that regulation oftox expression Is mediated through an iron-binding aporepressor. To facilitate molecular cloning of the tox regulatory element from genomic libraries of C. diphtheriae, we constructed a tox promoter/operator (toxPO)-acZ transcriptional fusion in Escherichia coli strain DH5a. We report the molecular cloning and nucleic acid sequence of a diphtheria tox iron-dependent regulatory element, dxR, and demonstrate that expression of f-galactosidase from the toxPO-4acZ fusion is regulated by dlxR-encoded protein in an iron-sensitive manner. In addition, we show that expression of the toxPO-acZ fusion is not affected by the E. coUl ironregulatory protein Fur and that the dlxR protein does not inhibit expression offur-regulated outer-membrane proteins.Diphtheria toxin is synthesized by Corynebacterium diphtheriae lysogenic for one of a family of corynebacteriophages that carries the structural gene for the toxin, tox (1, 2). Optimal yields of tox gene products have long been known to be obtained only from C. diphtheriae grown under conditions where iron becomes the growth-rate-limiting substrate (3, 4). In 1936, Pappenheimer and Johnson (5) showed that adding iron in low concentration to the growth medium inhibited the production of diphtheria toxin. Both biochemical and genetic evidence support the hypothesis that the corynebacteriophage tox gene is regulated by a corynebacterial-determined iron-binding repressor as postulated by . This model predicted an aporepressor that in the presence of iron forms a complex; this complex then binds to the tox operator and blocks transcription. Under conditions of iron limitation, the iron-repressor complex dissociates, derepressing the tox gene.The nucleic acid base sequence of tox revealed a 9-basepair (bp) inverted repeat that overlaps the " -10" region of the promoter (13). Because many operators exhibit dyad symmetry and are positioned near their respective promoters, this region was designated the putative tox operator.We here describe the genetic construction of an Escherichia coli host strain that carries a chromosomal diphtheria tox promoter/operator (toxPO)-lacZ transcriptional fusion in single copy. Because this strain constitutively expresses j-galactosidase and is phenotypically blue on 5-bromo-4-chloro-3-indolyl /3-D-galactoside (X-Gal)-containing agar medium, we have used it to screen genomic libraries of nontoxinogenic nonlysogenic C. diphtheriae for determinants that repress lacZ expression. We report the molecular cloning and deduced amino acid sequence (25,316 molecular weight) of a diphtheria tox iron-dependent regulatory element, dtxR.* We show that this factor acts as a negati...