Structurally colored materials are often used for their resistance to photobleaching and their complex viewing-direction-dependent optical properties. Frequently, absorption has been added to these types of materials in order to improve the color saturation by mitigating the effects of nonspecific scattering that is present in most samples due to imperfect manufacturing procedures. The combination of absorbing elements and structural coloration often yields emergent optical properties. Here, a new hybrid architecture is introduced that leads to an interesting, highly directional optical effect. By localizing absorption in a thin layer within a transparent, structurally colored multilayer material, an optical Janus effect is created, wherein the observed reflected color is different on one side of the sample than on the other. A systematic characterization of the optical properties of these structures as a function of their geometry and composition is performed. The experimental studies are coupled with a theoretical analysis that enables a precise, rational design of various optical Janus structures with highly controlled color, pattern, and fabrication approaches. These asymmetrically colored materials will open applications in art, architecture, semitransparent solar cells, and security features in anticounterfeiting materials.