We provide a short-time large deviation principle (LDP) for stochastic volatility models, where the volatility is expressed as a function of a Volterra process. This LDP holds under suitable conditions, but does not require any self-similarity assumption on the Volterra process. For this reason, we are able to apply such LDP to two notable examples of non self-similar rough volatility models: models where the volatility is given as a function of a log-modulated fractional Brownian motion [Bayer et al., Log-modulated rough stochastic volatility models. SIAM J. Financ. Math, 2021Math, , 12(3), 1257Math, -1284, and models where it is given as a function of a fractional Ornstein-Uhlenbeck (fOU) process [Gatheral et al., Volatility is rough. Quant. Finance, 2018, 18(6), 933-949]. In both cases we derive consequences for short-maturity European option prices and implied volatility surfaces. In the fOU case we also discuss moderate deviations pricing and simulation results.