Color perception relies on spatial comparisons of chromatic signals, but how the brain performs these comparisons is poorly understood. Here, we show that many V1 neurons compare signals across their receptive fields (RF) using a common principle. We estimated the spatial-chromatic RFs of each neuron, and then measured neural selectivity to customized colored edges that were sampled using a novel closed-loop technique. We found that many double-opponent (DO) cells, which have spatially and chromatically opponent RFs, responded to chromatic contrast as a weighted sum, akin to how simple cells respond to luminance contrast. Other neurons integrated chromatic signals non-linearly, confirming that linear signal integration is not an obligate property of V1 neurons. The functional similarity of DO and simple cells suggests a common underlying neural circuitry, promotes the construction of image-computable models for full-color image representation, and sheds new light on V1 complex cells.