We study a hybrid structure of a ferromagnetic-insulator and a superconductor connected by a weak link, which accommodates Andreev bound states whose spin degeneracy is lifted due to the exchange interaction with the ferromagnet. The resultant spin-resolved energy levels realize a twostate quantum system, provided that a single electron is trapped in the bound state, i.e., an Andreev spin qubit. The qubit state can be manipulated by controlling the magnetization dynamics of the ferromagnet, which mediates the coupling between external fields and the qubit. In particular, our hybrid structure provides a simple platform to manipulate and control the spin qubit using spintronic techniques. By employing a modified Hahn spin echo protocol for the magnetization dynamics, we show that our Andreev spin qubit can realize a nonadiabatic geometric gate.