Geometric phases are a universal concept that underpins numerous phenomena involving multicomponent wave fields. These polarization-dependent phases are inherent in interference effects, spin-orbit interaction phenomena, and topological properties of vector wave fields. Geometric phases have been thoroughly studied in two-component fields, such as two-level quantum systems or paraxial optical waves. However, their description for fields with three or more components, such as generic nonparaxial optical fields routinely used in modern nano-optics, constitutes a nontrivial problem.Here we describe geometric, dynamical, and total phases calculated along a closed spatial contour in a multi-component complex field, with particular emphasis on 2D (paraxial) and 3D (nonparaxial) optical fields. We present several equivalent approaches: (i) an algebraic formalism, universal for any multi-component field; (ii) a dynamical approach using the Coriolis coupling between the spin angular momentum and reference-frame rotations; and (iii) a geometric representation, which unifies the Pancharatnam-Berry phase for the 2D polarization on the Poincaré sphere and the Majoranasphere representation for the 3D polarized fields. Most importantly, we reveal close connections between geometric phases, angular-momentum properties of the field, and topological properties of polarization singularities in 2D and 3D fields, such as C-points and polarization Möbius strips.