We investigate the joint dynamics of spot and implied volatility from an empirical perspective. We focus on the equity market with the SPX Index our underlying of choice. Using only observable quantities, we extract the instantaneous variance curves implied by the market and study their daily variations jointly with spot returns. We analyze the characteristics of their individual and joint densities, quantify the non-linear relationship between spot and volatility, and discuss the modeling implications on the implied leverage and the volatility clustering effects. We show that non-linearities have little impact on the dynamics of at-the-money volatilities, but can have a significant effect on the pricing and hedging of volatility derivatives.