Competing rivalrous neural representations can be resolved at several levels of the visual system. Sustained percepts during interocular-switch rivalry (ISR), in which rivalrous left- and right-eye stimuli swap between eyes several times a second, often are attributed to competing binocularly driven neural representations of each rivalrous stimulus. An alternative view posits monocular neural competition together with a switch in eye dominance at the moment of each stimulus swap between eyes. Here, a range of experimental conditions was tested that would change the colors seen if mediated by eye dominance but not if by competition between binocularly driven responses. Observers viewed multiple chromatically rivalrous discs in various temporal and spatial patterns, and reported when all discs in view appeared the same color. Unlike typical ISR paradigms that swap the complete stimulus in each eye, some of the rivalrous discs were swapped at a different time, or faster frequency, than other discs. Monocular dominance of one eye at a time implies that all discs will rarely be seen as identical in color when some discs swap at a different frequency than others. On the other hand, competing binocularly driven neural responses are not affected by asynchronous swap timing among the individual discs. Results for every observer are in accord with competing responses at the level of binocularly driven, chromatically tuned neurons. Although an account based on eye dominance can be constructed using many small retinotopic zones that have independent timing for the moment of switching the dominant eye, competing binocularly driven responses are a more parsimonious explanation.