Optically pumped nonlinear frequency down conversion is a proven approach for monochromatic terahertz (THz)-wave generation that provides superior properties such as continuous and wide tunability as well as laser-like linewidth and beam quality. Phase-matching (PM) is an important connection between the pump sources and nonlinear crystals and determines the direction of energy flow (as well as the output power). In past decades, a variety of peculiar PM configurations in the THz region have been invented and are different from the traditional ones in the optical region. We summarize the configurations that have been applied in nonlinear THz-wave generation, which mainly fall in two categories: scalar (collinear) PM and vector PM (including macroscopic noncollinear PM and microscopic vector PM). The development of this technique could relax the matching conditions in a wide range of nonlinear crystals and pump wavelengths and could finally promote the improvement of coherent THz sources.