Colloidal quantum dot solar cells achieve spectrally selective optical absorption in a thin layer of solution-processed, size-effect tuned, nanoparticles. The best devices built to date have relied heavily on drift-based transport due to the action of an electric field in a depletion region that extends throughout the thickness of the quantum dot layer. Here we study for the first time the behaviour of the best-performing class of colloidal quantum dot films in the absence of an electric field, by screening using an electrolyte. We find that the action of selective contacts on photovoltage sign and amplitude can be retained, implying that the contacts operate by kinetic preferences of charge transfer for either electrons or holes. We develop a theoretical model to explain these experimental findings. The work is the first to present a switch in the photovoltage in colloidal quantum dot solar cells by purposefully formed selective contacts, opening the way to new strategies in the engineering of colloidal quantum dot solar cells.