We consider localized qubits evolving around a black hole following a quantum adiabatic dynamics. We develop a geometric structure (based on fibre bundles) permitting to describe the quantum states of a qubit and the spacetime geometry in a single framework. The quantum decoherence induced by the black hole on the qubit is analysed in this framework (the role of the dynamical and geometric phases in this decoherence is treated), especially for the quantum teleportation protocol when one qubit falls to the event horizon. A simple formula to compute the fidelity of the teleportation is derived. The case of a Schwarzschild black hole is analysed.