We give an overview of recent exciting achievements of deep reinforcement learning (RL). We discuss six core elements, six important mechanisms, and twelve applications. We start with background of machine learning, deep learning and reinforcement learning. Next we discuss core RL elements, including value function, in particular, Deep Q-Network (DQN), policy, reward, model and planning, exploration, and knowledge. After that, we discuss important mechanisms for RL, including attention and memory, unsupervised learning, transfer learning, multiagent RL, hierarchical RL, and learning to learn. Then we discuss various applications of RL, including games, in particular, AlphaGo, robotics, natural language processing, including dialogue systems, machine translation, and text generation, computer vision, business management, finance, healthcare, education, Industry 4.0, smart grid, intelligent transportation systems, and computer systems. We mention topics not reviewed yet, and list a collection of RL resources. After presenting a brief summary, we close with discussions. This is the first overview about deep reinforcement learning publicly available online. It is comprehensive. Comments and criticisms are welcome. (This particular version is incomplete.)Please see Deep Reinforcement Learning, https://arxiv.org/abs/ 1810.06339, for a significant update to this manuscript.

China's Chang'E-3 (CE-3) spacecraft touched down on the northern Mare Imbrium of the lunar nearside (340.49°E, 44.12°N), a region not directly sampled before. We report preliminary results with data from the CE-3 lander descent camera and from the Yutu rover's camera and penetrating radar. After the landing at a young 450-meter crater rim, the Yutu rover drove 114 meters on the ejecta blanket and photographed the rough surface and the excavated boulders. The boulder contains a substantial amount of crystals, which are most likely plagioclase and/or other mafic silicate mineral aggregates similar to terrestrial dolerite. The Lunar Penetrating Radar detection and integrated geological interpretation have identified more than nine subsurface layers, suggesting that this region has experienced complex geological processes since the Imbrian and is compositionally distinct from the Apollo and Luna landing sites.

Polyploidy can lead to aneuploidy and tumorigenesis. Here, we report that the Hippo pathway effector Yap promotes the diploid-polyploid conversion and polyploid cell growth through the Akt-Skp2 axis. Yap strongly induces the acetyltransferase p300-mediated acetylation of the E3 ligase Skp2 via Akt signaling. Acetylated Skp2 is exclusively localized to the cytosol, which causes hyper-accumulation of the cyclin-dependent kinase inhibitor p27, leading to mitotic arrest and subsequently cell polyploidy. Additionally, the pro-apoptotic factors FoxO1/3 are overly degraded by acetylated Skp2, resulting in polyploid cell division, genomic instability and oncogenesis. Importantly, the depletion or inactivation of Akt or Skp2 abrogated Hippo signal deficiency-induced liver tumorigenesis, indicating their epistatic interaction. Thus, we conclude that Hippo-Yap signaling suppresses cell polyploidy and oncogenesis through Skp2.

Objective. To study the osteogenic differentiation capacity of bone marrow-derived mesenchymal stem cells (BM-MSCs) from patients with ankylosing spondylitis (AS) and to investigate the mechanisms of abnormal osteogenic differentiation of BM-MSCs in AS.Methods. BM-MSCs from healthy donors (HDMSCs) and patients with AS (AS-MSCs) were cultured in osteogenic differentiation medium for 0-21 days, after which their osteogenic differentiation capacity was determined using alizarin red S and alkaline phosphatase assays. Gene expression levels of osteoblastic markers and related cytokines were detected by high-throughput quantitative reverse transcription-polymerase chain reaction. Enzyme-linked immunosorbent assay was performed to detect protein levels of bone morphogenetic protein 2 (BMP-2) and Noggin in the cell culture supernatant. The activation of Smad1/5/8 and MAPK signaling pathways was measured by Western blotting. The balance between BMP-2 and Noggin expression was regulated using lentiviruses encoding short hairpin RNA and exogenous Noggin, respectively, which enabled evaluation of how this balance affected osteogenic differentiation of AS-MSCs.Results. AS-MSCs outperformed HD-MSCs in osteogenic differentiation capacity. During osteogenic differentiation, AS-MSCs secreted more BMP-2 but less Noggin, accompanied by an overactivation of Smad1/5/ 8 and ERK-1/2. When the Noggin concentration was increased or BMP-2 expression was inhibited, the abnormal osteogenic differentiation of AS-MSCs was rectified. In addition, the balance between BMP-2 and Noggin secretion was restored.Conclusion. The results of this study demonstrate that an imbalance between BMP-2 and Noggin secretion induces abnormal osteogenic differentiation of AS-MSCs. These findings reveal a mechanism of pathologic osteogenesis in AS and provide a new perspective on inhibiting pathologic osteogenesis by regulating the balance between BMP-2 and Noggin.