Objective. To highlight the unique challenges of evaluative research on practice behavior change in the “real world'' settings of contemporary managed‐care organizations, using the experience of the Pediatric Asthma Care PORT (Patient Outcomes Research Team). Study Setting. The Pediatric Asthma Care PORT is a five‐year initiative funded by the Agency for Healthcare Research and Quality to study strategies for asthma care improvement in three managed‐care plans in Chicago, Seattle, and Boston. At its core is a randomized trial of two care improvement strategies compared with usual care: (1) a targeted physician education program using practice based Peer Leaders (PL) as change agents, (2) adding to the PL intervention a “Planned Asthma Care Intervention'' incorporating joint “asthma check‐ups'' by nurse‐physician teams. During the trial, each of the participating organizations viewed asthma care improvement as an immediate priority and had their own corporate improvement programs underway. Data Collection. Investigators at each health plan described the organizational and implementation challenges in conducting the PAC PORT randomized trial. These experiences were reviewed for common themes and “lessons'' that might be useful to investigators planning interventional research in similar care‐delivery settings. Conclusions. Randomized trials in “real world'' settings represent the most robust design available to test care improvement strategies. In complex, rapidly changing managed‐care organizations, blinding is not feasible, corporate initiatives may complicate implementation, and the assumption that a “usual care'' arm will be static is highly likely to be mistaken. Investigators must be prepared to use innovative strategies to maintain the integrity of the study design, including: continuous improvement within the intervention arms, comanagement by researchers and health plan managers of condition‐related quality improvement initiatives, procedures for avoiding respondent burden in health plan enrollees, and anticipation and minimization of risks from experimental arm contamination and major organizational change. With attention to these delivery system issues, as well as the usual design features of randomized trials, we believe managed‐care organizations can serve as important laboratories to test care improvement strategies.