While a number of algorithms for multiobjective reinforcement learning have been proposed, and a small number of applications developed, there has been very little rigorous empirical evaluation of the performance and limitations of these algorithms. This paper proposes standard methods for such empirical evaluation, to act as a foundation for future comparative studies. Two classes of multiobjective reinforcement learning algorithms are identified, and appropriate evaluation metrics and methodologies are proposed for each class. A suite of benchmark problems with known Pareto fronts is described, and future extensions and implementations of this benchmark suite are discussed. The utility of the proposed evaluation methods are demonstrated via an empirical comparison of two example learning algorithms.

Background Identifying differences and similarities between CLA+ polarized T-cell subsets in pediatric vs. adult atopic dermatitis (AD) is critical for directing new treatments towards children. Objective To compare activation markers and frequencies of skin-homing (CLA+) vs. systemic (CLA−) “polar” CD4 and CD8 T-cell subsets in early pediatric AD, adult AD and controls. Methods Flow cytometry was used to measure CD69/ICOS/HLA-DR frequency in memory subsets as well as IFN-γ, IL-13, IL-9, IL-17, and IL-22 cytokines, defining Th1/Tc1, Th2/Tc2, Th9/Tc9, Th17/Tc17, and Th22/Tc22 populations in CD4 and CD8 cells, respectively. We compared peripheral blood from 19 children <5 years and 42 adults with well-characterized moderate to severe AD, as well as age-matched controls (17 children and 25 adults). Results Selective ICOS activation (P<0.001) was seen in children. CLA+ Th2 T-cells were markedly expanded in both AD children and adults compared to controls, but decreases in CLA+ Th1 T-cells were greater in AD children (17% vs. 7.4%, P=0.007). Unlike in adults, no imbalances were detected in pediatric AD CLA− T-cells, nor were there altered frequencies of Th22 T-cells within CLA+ or CLA− compartments. Adults with AD had increased frequency of IL-22-producing CD4 and CD8 T-cells within the skin-homing population (9.5% vs. 4.5%; 8.6% vs. 2.4%, respectively; P<0.001) as well as increased HLA-DR activation (P<0.01). Conclusions These data suggest that Th2 activation within skin-homing T-cells may drive AD in children and that reduced counter-regulation by Th1 T-cells may contribute to excess Th2 activation. Th22 “spreading” of AD is not seen in young children and may be influenced by immune development, disease chronicity or recurrent skin infections. Clinical Implications Therapeutic approaches to treat AD in children may be best directed to correction of Th2/Th1 imbalance, while adults might also benefit from IL-22 targeted therapies.

Vascular endothelial growth factor (VEGF) has been identified as a vascular permeability factor, angiogenic cytokine, and a survival factor. To address its role in mammary carcinogenesis, we used transgenic mice with human VEGF(165) targeted to mammary epithelial cells under the control of the mouse mammary tumor virus (MMTV) promoter. Metastatic mammary carcinomas were induced by mating the MMTV-VEGF mice with MMTV-polyoma virus middle T-antigen (MT) mice to generate VEGF/MT mice. Tumor latency was decreased in the VEGF/MT mice, which developed mammary carcinomas with increased vasodilatation at 4 weeks of age. There was increased incidence, multiplicity, and weight of the mammary tumors in 6- and 8-week-old VEGF/MT mice, compared to their MT-only littermates. Macro- and microscopic lung metastases were detected in the VEGF/MT mice but not the MT mice at 6 and 8 weeks of age. Enhanced tumor growth was attributed to increased microvascular density (MVD), as well as increased tumor cell proliferation and survival. Angiogenesis array analysis showed that 24 of 25 differentially expressed genes were upregulated in the VEGF/MT tumors. In vitro studies revealed increased proliferative activity and upregulation of Flk-1 in the VEGF/MT tumor cells, compared with the MT-only tumor cells. Moreover, there was decreased proliferative activity with downregulation of Flk-1 in tumor cells isolated from conditional knockout (VEGF(-/-)) MT-induced mammary carcinomas. The slow growing VEGF(-/-) tumor cells were accumulated in the G(1)/G(0) phase of the cell cycle and this was associated with stimulation of p16(ink4a) and p21(WAF1). Similarly, p16(ink4a) was stimulated in VEGF(lox/lox)/MT mammary tumor cells following Adeno-cre-mediated VEGF gene inactivation. Collectively, the data from these transgenic models indicate that VEGF contributes to mammary tumor growth through increased neovascularization, as well as autocrine stimulation of growth and inhibition of apoptosis.