We introduce the passive all-optical polarization switch, which modulates light with light. That switch is used to construct all the binary logic gates of two or more inputs. We discuss the design concepts and the operation of the AND, OR, NAND, and NOR gates as examples. The rest of the 16 logic gates are similarly designed. Cascading of such gates is straightforward as we show and discuss. Cascading in itself does not require a power source, but feedback at this stage of development does. The design and operation of an SR Latch is presented as one of the popular basic sequential devices used for memory cells. That completes the essential components of an all-optical polarization digital processor. The speed of such devices is well above 10 GHz for bulk implementations and is much higher for chip-size implementations. In addition, the presented devices do have the four essential characteristics previously thought unique to the microelectronic ones.