We provide a framework for non-relativistic holography so that a covariant action principle ensuring the Galilean symmetry for dual conformal field theory is given. This framework is based on the Bargmann lift of the Newton-Cartan gravity to the one-dimensional higher Einstein gravity, or reversely, the null-like Kaluza-Klein reduction. We reproduce the previous zero temperature results, and our framework provides a natural explanation about why the holography is co-dimension 2. We then construct the black hole solution dual to the thermal CFT, and find the horizon is curvature singular. However, we are able to derive the sensible thermodynamics for the dual non-relativistic CFT with correct thermodynamical relations. Besides, our construction admits a null Killing vector in the bulk such that the Galilean symmetry is preserved under the holographic RG flow. Finally, we evaluate the viscosity and find it zero if we neglect the back reaction of the singular horizon, otherwise, it could be nonzero. 1