Motivation: Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora. Results: We introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62% F1 score improvement), biomedical relation extraction (2.80% F1 score improvement) and biomedical question answering (12.24% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts. Availability and implementation: We make the pre-trained weights of BioBERT freely available at https://github. com/naver/biobert-pretrained, and the source code for fine-tuning

Lithium-ion batteries are now reaching the energy density limits set by their electrode materials, requiring new paradigms for Li(+) and electron hosting in solid-state electrodes. Reversible oxygen redox in the solid state in particular has the potential to enable high energy density as it can deliver excess capacity beyond the theoretical transition-metal redox-capacity at a high voltage. Nevertheless, the structural and chemical origin of the process is not understood, preventing the rational design of better cathode materials. Here, we demonstrate how very specific local Li-excess environments around oxygen atoms necessarily lead to labile oxygen electrons that can be more easily extracted and participate in the practical capacity of cathodes. The identification of the local structural components that create oxygen redox sets a new direction for the design of high-energy-density cathode materials.

Nearly all high-energy density cathodes for rechargeable lithium batteries are well-ordered materials in which lithium and other cations occupy distinct sites. Cation-disordered materials are generally disregarded as cathodes because lithium diffusion tends to be limited by their structures. The performance of Li1.211Mo0.467Cr0.3O2 shows that lithium diffusion can be facile in disordered materials. Using ab initio computations, we demonstrate that this unexpected behavior is due to percolation of a certain type of active diffusion channels in disordered Li-excess materials. A unified understanding of high performance in both layered and Li-excess materials may enable the design of disordered-electrode materials with high capacity and high energy density.

There is an urgent need for low-cost, resource-friendly, high-energy-density cathode materials for lithium-ion batteries to satisfy the rapidly increasing need for electrical energy storage. To replace the nickel and cobalt, which are limited resources and are associated with safety problems, in current lithium-ion batteries, high-capacity cathodes based on manganese would be particularly desirable owing to the low cost and high abundance of the metal, and the intrinsic stability of the Mn oxidation state. Here we present a strategy of combining high-valent cations and the partial substitution of fluorine for oxygen in a disordered-rocksalt structure to incorporate the reversible Mn/Mn double redox couple into lithium-excess cathode materials. The lithium-rich cathodes thus produced have high capacity and energy density. The use of the Mn/Mn redox reduces oxygen redox activity, thereby stabilizing the materials, and opens up new opportunities for the design of high-performance manganese-rich cathodes for advanced lithium-ion batteries.