We present neutron scattering measurements on single crystals of lightly doped La2−xBaxCuO4, with 0 ≤ x ≤ 0.035. These reveal the evolution of the magnetism in this prototypical doped Mott insulator from a three dimensional (3D) commensurate (C) antiferromagnetic ground state, which orders at a relatively high TN , to a two dimensional (2D) incommensurate (IC) ground state with finite ranged static correlations, which appear below a relatively low effective TN . At low temperatures, the 2D IC magnetism co-exists with the 3D C magnetism for doping concentrations as low as ∼ 0.0125. We find no signal of a 3D C magnetic ground state by x ∼ 0.025, consistent with the upper limit of x ∼ 0.02 observed in the sister family of doped Mott insulators, La2−xSrxCuO4. The 2D IC ground states observed for 0.0125 ≤ x ≤ 0.035 are diagonal, and are rotated by 45 degrees within the orthorhombic basal plane compared with those previously reported for samples with superconducting ground states: La2−xBaxCuO4, with 0.05 ≤ x ≤ 0.095. We construct a phase diagram based solely on magnetic order parameter measurements, which displays much of the complexity of standard high temperature superconductivity phase diagrams discussed in the literature. Analysis of high energy-resolution inelastic neutron scattering at moderately low temperatures shows a progressive depletion of the very low energy dynamic magnetic susceptibility as x increases from 0.0125 to 0.035. This low energy, dynamic susceptibility falls off with increasing temperature on a scale much higher than the effective 2D IC TN appropriate to these materials. Appreciable dynamic 2D IC magnetic fluctuations inhabit much of the "pseudogap" regime of the phase diagram.