In five experiments, we examined task factors influencing selection by color in a rapid serial visual presentation (RSVP) task. Participants were required to respond to the presence or case of a target letter presented within an RSVP sequence, and they were cued as to the color in which that target was most likely to occur. The first two experiments explored the effect of task difficulty. It was found that a change from the discrimination of upper- versus lowercase targets to the detection of an exact shape had no influence on the color-cuing effect. In contrast, increasing letter set discriminability and decreasing color discriminability reduced the color-cuing effect considerably, although they did not eliminate it. In the third and fourth experiments, the frequency of valid trials was reduced, removing the incentive to use the color information. Again, a small benefit remained, indicating partially automatic and strategic color-processing components. Finally, the fifth experiment showed that an increase in perceptual load had no influence on the color-cuing effect. Together, these five experiments provide further evidence of the robustness of direct selection by color.