The magnetic dipole radiation (MDR) model is currently the best approach we have to explain pulsar radiation. However a most characteristic parameter of the observed radiation, the braking index n obs shows deviations for all the eight best studied isolated pulsars, from the simple model prediction n dip = 3. The index depends upon the rotational frequency and its first and second time derivatives, but also on the assumption of that the magnetic dipole moment and inclination angle, and the moment of inertia of the pulsar are constant in time. In a recent paper [Phys. Rev. D 91, 063007 (2015)] we showed conclusively that changes in the moment of inertia with frequency alone, cannot explain the observed braking indices.Possible observational evidence for the magnetic dipole moment migrating away from the rotational axis at a rateα ∼ 0.6 • per 100 years over the life time of the Crab pulsar has been recently suggested by Lyne et al. In this paper, we explore the MDR model with constant moment of inertia and magnetic dipole moment but variable inclination angle α. We first discuss the effect of the variation of α on the observed braking indices and show they all can be understood. However, no explanation for the origin of the change in α is provided.After discussion of the possible source(s) of magnetism in pulsars we propose a simple mechanism for the change in α based on a toy model in which the magnetic structure in pulsars consists of two interacting dipoles. We show that such a system can explain the Crab observation and the measured braking indices.