Pulsed laser ablation in liquids (PLAL) is a promising technique for the generation of colloidal alloy nanoparticles that are of high demand in a broad range of fields, including catalysis, additive manufacturing, and biomedicine. Many of the applications have stringent requirements on the nanoparticle composition and size distributions, which can only be met through innovations in the PLAL technique guided by a clear understanding of the nanoparticle formation mechanisms. In this work, we undertake a combined computational and experimental study of the nanoparticle formation mechanisms in ultrashort PLAL of Ag/Cu and Cu/Ag bilayer thin films. Experimental probing of the composition of individual nanoparticles and predictions from large-scale atomistic simulations provide consistent evidence of limited mixing between the two components from bilayer films by PLAL. The simulated and experimental distributions of nanoparticle compositions exhibit an enhanced abundance of Ag-rich and Cu-rich nanoparticles, as well as a strongly depressed population of well-mixed alloy nanoparticles. The surprising observation that the nanoscale phase separation of the two components in the bilayer films manifests itself in the sharp departure from the complete quantitative mixing in the colloidal nanoparticles is explained by the complex dynamic interaction between the ablation plume and liquid environment revealed in the simulations of the initial stage of the ablation process. The simulations predict that rapid deceleration of the ablation plume by the liquid environment results in the formation of a transient hot and dense metal region at the front of the plume, which hampers the mixing of the two components and, at the same time, contributes to the stratification of the plume in the emerging cavitation bubble. As a result, nanoparticles of different sizes and compositions are produced in different parts of the emerging cavitation bubble during the first nanoseconds of the ablation process. Notably, the diameters of the largest nanoparticles generated in the simulations of the initial stage of the ablation process are more than twice larger than the thickness of the original bilayer films. This observation provides a plausible scenario for the formation of large nanoparticles observed in the experiments. The conclusion on limited elemental mixing in the nanoparticles is validated in simulations of bilayers with different spatial order of Cu and Ag layers, even though the two systems exhibit some notable quantitative differences mainly related to the different strength of electron−phonon coupling in Cu and Ag. Overall, the results of this study provide new insights into the formation mechanism of bimetallic nanoparticles in ultrashort PLAL from thin bilayer targets and suggest that the formation of alloy nanoparticles from immiscible elements may be hampered for targets featuring distinctive elemental segregation.