Cross-nucleation is a phenomenon where a new crystal nucleates and grows upon the surface of a different polymorph. Previous studies indicate that faster growth rate of the new crystal is a necessary but not sufficient condition for cross-nucleation. The thermodynamic stability of the different polymorphs can also affect cross-nucleation by modulating the rates of crystal growth. The interplay between thermodynamic stability of the polymorphs involved, the growth rate of the crystals, and the need for creation of an interfacial transition layer that seamlessly connects the two structures has not yet been fully elucidated. Predicting cross-nucleation is particularly challenging for clathrate hydrates, for which there are sometimes several polymorphs with similar stability and for which growth rates are not known. In this work, we use molecular dynamics simulations to investigate which factor (stability, growth rate, or formation of interfacial transition layer) controls cross-nucleation between the four known Frank-Kasper clathrate hydrate polymorphs: sI, sII, TS, and HS-I. We investigate the growth and cross-nucleation of these four hydrates filled with a set of guest molecules that produce different order of stabilities for the four crystal structures. We determine that the growth rate of sII clathrate is the fastest, followed by TS, HS-I, and sI. We find that cross-nucleation into or from sII clathrates is preceded by the formation of an interfacial transition layer at the seed crystal/liquid interface because sII does not share a crystal plane with sI, HS-I, or TS. Cross-nucleation between the latter three can occur seamlessly and is determined only by their growth rates. Our results indicate that nucleation of an interfacial transition layer between non-matching polymorphs can control cross-nucleation or lack thereof under conditions of small driving force. Under conditions of sufficient supercooling clathrate hydrate polymorphs cross-nucleate into the fastest growing phase even if that new phase is less stable and does not share a common crystal plane with the initial polymorph.