Storm has long served as the main platform for real-time analytics at Twitter. However, as the scale of data being processed in realtime at Twitter has increased, along with an increase in the diversity and the number of use cases, many limitations of Storm have become apparent. We need a system that scales better, has better debug-ability, has better performance, and is easier to manage -all while working in a shared cluster infrastructure. We considered various alternatives to meet these needs, and in the end concluded that we needed to build a new real-time stream data processing system. This paper presents the design and implementation of this new system, called Heron. Heron is now the de facto stream data processing engine inside Twitter, and in this paper we also share our experiences from running Heron in production. In this paper, we also provide empirical evidence demonstrating the efficiency and scalability of Heron.

Predators and dry-disturbances have pronounced effects on invertebrate communities and can deterministically affect compositional turnover between discrete aquatic habitats. We examined the effect of sunfish (Lepomis spp.) predators on two native crayfish, Procambarus alleni and P. fallax, that regionally coexist in an expansive connected wetland in order to test the hypotheses that sunfish predation limits crayfish recruitment (both species) and that it disproportionately affects P. alleni, the species inhabiting temporary wetlands. In replicate vegetated wetlands (18.6 m(2)), we quantified summertime crayfish recruitment and species composition across an experimental gradient of sunfish density. Separately, we quantified effects of sunfish on crayfish growth, conducted a complimentary predation assay in mesocosms, and compared behavior of the two crayfish. Sunfish reduced P. alleni summertime recruitment by >99% over the full sunfish gradient, and most of the effect was caused by low densities of sunfish (0.22-0.43 m(-2)). P. alleni dominated wetlands with few or no sunfish, but the composition shifted towards P. fallax dominance in wetlands with abundant sunfish. P. fallax survived better than P. alleni over 40 h in smaller mesocosms stocked with warmouth. Sunfish reduced P. fallax recruitment 62% in a second wetland experiment, but growth rates of caged crayfish (both species) were unaffected by sunfish presence, suggesting predatory effects were primarily consumptive. Consistent with life histories of relatively fish-sensitive invertebrates, P. alleni engaged in more risky behaviors in the laboratory. Our results indicate that sunfish predators limit recruitment of both species, but disproportionately remove the more active and competitively dominant P. alleni. Spatially and temporally variable dry-disturbances negatively co-varying with sunfish populations allow for regional coexistence of these two crayfish and may release populations of either species from control by predatory fishes.

Despite the large number of available studies, most women with breast cancer do not participate in clinical trials, and this is especially true among lower income and minority women. In this study the authors surveyed the practice patterns of four medical oncologists who comprised the clinical breast service at a large urban university hospital to develop a better understanding of the clinical trials enrollment process for women with breast cancer. Of 136 new female breast cancer patients seen by the four physicians over a 7-month period, there were 47 women (34%) offered participation in a clinical trial, and 16 women (12%) were eventually enrolled. Women who were offered participation were more likely to be younger (p = 0.068) and to have earlier stage disease then were women not offered participation (p = 0.008). Women enrolled into a trial were also more likely to be younger, although this difference was not statistically significant (p = 0.114). Patient race was not associated with the accrual or enrollment process. Over half of the women were not offered participation in clinical trials because of the lack of available studies. Further work evaluating the process of patient enrollment and physician and patient barriers is necessary to develop effective strategies for recruitment into breast cancer clinical trials.