The temperature dependence of quantum Hall conductivities is studied in the context of the AdS/CMT paradigm using a model with a bulk theory consisting of (3+1)-dimensional Einstein-Maxwell action coupled to a dilaton and an axion, with a negative cosmological constant. We consider a solution which has a Lifshitz like geometry with a dyonic black-brane in the bulk. There is an Sl(2,R) action in the bulk corresponding to electromagnetic duality, which maps between classical solutions, and is broken to Sl(2,Z) by Dirac quantisation of dyons. This bulk Sl(2,Z) action translates to an action of the modular group on the 2-dimensional transverse conductivities. The temperature dependence of the infra-red conductivities is then linked to modular forms via gradient flow and the resulting flow diagrams show remarkable agreement with existing experimental data on the temperature flow of both integral and fractional quantum Hall conductivities.