The geometric phase of light has been demonstrated in various platforms of the linear optical regime, raising interest both for fundamental science as well as applications, such as flat optical elements. Recently, the concept of geometric phases has been extended to nonlinear optics, following advances in engineering both bulk nonlinear photonic crystals and nonlinear metasurfaces. These new technologies offer a great promise of applications for nonlinear manipulation of light. In this review, we cover the recent theoretical and experimental advances in the field of geometric phases accompanying nonlinear frequency conversion. We first consider the case of bulk nonlinear photonic crystals, in which the interaction between propagating waves is quasi-phase-matched, with an engineerable geometric phase accumulated by the light. Nonlinear photonic crystals can offer efficient and robust frequency conversion in both the linearized and fully-nonlinear regimes of interaction, and allow for several applications including adiabatic mode conversion, electromagnetic nonreciprocity and novel topological effects for light. We then cover the rapidly-growing field of nonlinear Pancharatnam-Berry metasurfaces, which allow the simultaneous nonlinear generation and shaping of light by using ultrathin optical elements with subwavelength phase and amplitude resolution. We discuss the macroscopic selection rules that depend on the rotational symmetry of the constituent meta-atoms, the order of the harmonic generations, and the change in circular polarization. Continuous geometric phase gradients allow the steering of light beams and shaping of their spatial modes. More complex designs perform nonlinear imaging and multiplex nonlinear holograms, where the functionality is varied according to the generated harmonic order and polarization. Recent advancements in the fabrication of three dimensional nonlinear photonic crystals, as well as the pursuit of quantum light sources based on nonlinear metasurfaces, offer exciting new possibilities for novel nonlinear optical applications based on geometric phases.