This article is the second of a trilogy that addresses the perturbative response of general quantum systems, with possibly nontrivial ground state geometry, beyond linear order. Here, we establish concise, general formulae for second order response to a spatially uniform, time-varying electric field in the velocity gauge that are manifestly free of static limit spurious divergences. We first discuss general quantum evolution in a curved space, then detail how such a situation is a natural byproduct of Hilbert space truncation, and point out crucial subtleties associated with the resulting finite curvatures. We then present a geometric perspective of the two popular gauges often used in quantum transport theories, the velocity gauge and the length gauge, and discuss how they, taking truncation-induced curvature effects into account, naturally lead to the same results in spite of the truncation. We highlight subtle formal discrepancies in the literature. Finally, we provide a general scheme for removing static limit spurious divergences in the velocity gauge without frequency expansions and present concise and comprehensive Green's function formulae for responses up to second order. As an application of specific aspects of our theory, second order charge current responses in selected cases are analyzed in Refs. [3] and [63].