Solitonic symmetry is believed to follow the homotopy-group classification of topological solitons. Here, we point out a more sophisticated algebraic structure when solitons of different dimensions coexist in the spectrum. We uncover this phenomenon in a concrete quantum field theory, the 4d CP 1 model. This model has two kinds of solitonic excitations, vortices and hopfions, which would follow two U (1) solitonic symmetries according to homotopy groups. Nevertheless, we demonstrate the nonexistence of the hopfion U (1) symmetry by evaluating the hopfion charge of vortex operators. We clarify that what conserves hopfion numbers is a non-invertible symmetry generated by 3d spin topological quantum field theories (TQFTs). Its invertible subgroup is just Z2, which we recognize as a spin bordism invariant. Compared with the 3d CP 1 model, our work suggests a unified description of solitonic symmetries and couplings to topological phases.