We consider off-policy evaluation (OPE) of deterministic target policies for reinforcement learning (RL) in environments with continuous action spaces. While it is common to use importance sampling for OPE, it suffers from high variance when the behavior policy deviates significantly from the target policy. In order to address this issue, some recent works on OPE proposed in-sample learning with importance resampling. Yet, these approaches are not applicable to deterministic target policies for continuous action spaces. To address this limitation, we propose to relax the deterministic target policy using a kernel and learn the kernel metrics that minimize the overall mean squared error of the estimated temporal difference update vector of an action value function, where the action value function is used for policy evaluation. We derive the bias and variance of the estimation error due to this relaxation and provide analytic solutions for the optimal kernel metric. In empirical studies using various test domains, we show that the OPE with in-sample learning using the kernel with optimized metric achieves significantly improved accuracy than other baselines.

We previously demonstrated the safety and feasibility of mesenchymal stem cell (MSC) transplantation for bronchopulmonary dysplasia (BPD) in preterm infants in a phase I clinical trial. We thus investigated the therapeutic efficacy of MSCs for BPD in premature infants. A phase II double-blind, randomized, placebo-controlled clinical trial was conducted on preterm infants at 23 to 28 gestational weeks (GW) receiving mechanical ventilator support with respiratory deterioration between postnatal days 5 and 14. Infants were stratified by 23 to 24 GW and 25 to 28 GW and randomly allocated (1:1) to receive stem cells (1 × 10 7 cells/kg, n = 33) or placebo (n = 33). Although the inflammatory cytokines in the tracheal aspirate fluid were significantly reduced with MSCs, the primary outcome of death or severe/moderate BPD in the control group (18/33, 55%) was not significantly improved with MSC transplantation (17/33, 52%). In the subgroup analysis, the secondary outcome of severe BPD was significantly improved from 53% (8/15) to 19% (3/16) with MSC transplantation in the 23 to 24 GW group but not in the 25 to 28 GW subgroup. In summary, although MSC transplantation might be safe and feasible, this small study was underpowered to detect its therapeutic efficacy in preterm infants at 23 to 28 GW. Accordingly, we are now conducting an additional larger and controlled phase II clinical trial focusing on infants at 23 to 24 GW (NCT03392467). ClinicalTrials.gov identifier: NCT01828957.

This study retrospectively reviewed the medical records of children with lupus nephritis (LN) who were treated at Seoul National University Children's Hospital from 1986 to 2005 (mean duration 8.3+/-4.4 years). The records of 77 children (22 male and 55 female) were examined. The mean age at diagnosis was 11.9+/-3.0 years. The initial biopsy results revealed a WHO class IV classification for 60 (88.2%) of 68 biopsy proven cases. Of 77 patients, 67 (87.0%) responded initially to the high-dose corticosteroids with or without additional immunosuppressive therapy. Of the initial responders (67), 30 (44.8%) experienced at least one episode of proteinuric (24) or nephritic (6) flare. Thirteen patients (16.9%) progressed to either chronic renal failure (CRF) or end-stage renal disease (ESRD). Six (7.8%) patients died. A Kaplan-Meier estimate of patient survival and CRF-free survival rate was 95.4% and 88.7% at 5 years and 91.8% and 74.7% at 10 years, respectively. Multivariate analysis for class IV LN revealed male gender (P=0.029), initial hypertension (P=0.001) and absence of remission (P=0.002) to be prognostic factors predicting CRF. Glomerulosclerosis of 10% or more (P=0.005), nephritic flare (P=0.011), and presence of anti-phospholipid antibody (P=0.017) or syndrome (P=0.004) were also found to be independent risk factors for CRF. Cyclophosphamide pulse therapy failed to demonstrate superiority over other combined immunosuppressants used for the treatment of diffuse proliferative LN.

No abstract