The Darwin-Breit Hamiltonian is applied to the Aharonov-Bohm experiment. In agreement with the standard Maxwell-Lorentz theory, the force acting on electrons from infinite solenoids or ferromagnetic rods vanishes. However, the interaction energies and phase factors of the electron wave packets are non-zero. This allows us to explain the Aharonov-Bohm effect without involvement of electromagnetic potentials, fields, and topological properties of space.