Fig. 1. Overview of mix-and-match holography. We computationally design pairs of diffractive optical elements that encode multiple holograms or target images under different geometric alignments or DOE pairings (top left: encoding scheme, bottom left: fabricated DOEs). By changing the geometric alignment or pairing different DOEs, the individual encoded holograms can be de-multiplexed. Illumination with a pre-determined lighting condition produces the corresponding target image, that can be generalized in a wide range of scenarios (selected results presented on the right: through offset layer pairing (a), combinational pairing (b), multiview images' encoding (c), and animated images through linear translation (d)). The design process makes use of a combination of iterative phase retrieval methods and complex matrix factorization.Computational caustics and light steering displays offer a wide range of interesting applications, ranging from art works and architectural installations to energy efficient HDR projection. In this work we expand on this concept by encoding several target images into pairs of front and rear phasedistorting surfaces. Different target holograms can be decoded by mixing and matching different front and rear surfaces under specific geometric alignments. Our approach, which we call mix-and-match holography, is made possible by moving from a refractive caustic image formation process to a diffractive, holographic one. This provides the extra bandwidth that is required to multiplex several images into pairing surfaces.We derive a detailed image formation model for the setting of holographic projection displays, as well as a multiplexing method based on a combination of phase retrieval methods and complex matrix factorization. We demonstrate several application scenarios in both simulation and physical prototypes.