In this paper, we try to understand the pseudogap phenomenon observed in the cuprate superconductor through a model study. Specifically, we explore the socalled low-temperature pseudogap state by turning off the superconducting (SC) off diagonal long range order in an ansatz state for the t-J model (Weng 2011 New J. Phys. 13 103039). Besides strong non-Gaussian SC fluctuations, the resulting state also exhibits a systematic pseudogap behavior in both spin and charge degrees of freedom, manifested in the uniform spin susceptibility, specific heat, non-Drude resistivity, Nernst effect, as well as the quantum oscillation associated with small Fermi pockets emerging in strong magnetic fields, etc. These anomalous 'normal state' properties are found in qualitative consistency with experimental measurements in the cuprates. Such a model study establishes an intrinsic connection between the peculiar pseudogap properties and the non-BCS nature of the SC ground state. A critical comparison with other approaches to the doped Mott insulator is also made.