Context. We study the non-linear evolution of magnetic fields in neutron star crusts with special attention to the influence of the Hall drift. Aims. Our goal is to understand the conditions for fast dissipation due to the Hall term in the induction equation. We study the interplay of Ohmic dissipation and Hall drift in order to find a timescale for the overall crustal field decay. Methods. We solve numerically the Hall induction equation by means of a hybrid method (spectral in angles but finite differences in the radial coordinate). The microphysical input consists of the most modern available crustal equation of state, composition and electrical conductivities.Results. We present the first long term simulations of the non-linear magnetic field evolution in realistic neutron star crusts with a stratified electron number density and temperature dependent conductivity. We show that Hall drift influenced Ohmic dissipation takes place in neutron star crusts on a timescale of 10 6 years. When the initial magnetic field has magnetar strength, the fast Hall drift results in an initial rapid dissipation stage that lasts ∼ 10 4 years. The interplay of the Hall drift with the temporal variation and spatial gradient of conductivity tends to favor the displacement of toroidal fields toward the inner crust, where stable configurations can last for ∼ 10 6 years. We show that the thermally emitting isolated neutron stars, as the Magnificent Seven, are very likely descendants of neutron stars born as magnetars.