The local dynamics of ferroelectric domain polarization are uniquely investigated with sub-20-nm resolved maps of switching times, growth velocities, and growth directions. This is achieved by analyzing movies of hundreds of consecutive high speed piezo force microscopy images, which record domain switching dynamics through repeatedly alternating between high speed domain imaging and the application of 20-nanosecond voltage pulses. Recurrent switching patterns are revealed, and domain wall velocities for nascent domains are uniquely reported to be up to four times faster than for mature domains with radii greater than approximately 100 nm. Switching times, speeds, and directions are also shown to correlate with local mechanical compliance, with domains preferentially nucleating and growing in compliant sample regions while clearly shunting around locations with higher stiffness. This deterministic switching behavior strongly supports a defect-mediated energy landscape which controls polarization reversal, and that can therefore be predicted, modeled, and even manipulated through composition, processing, and geometry. Such results have important implications for the practical performance of ferroelectric devices by enabling guided optimization of switching times and feature densities, while the methods employed provide a new means to investigate and correlate dynamic functionality with mechanical properties at the nanoscale.