Context. The fast temporal evolution of the ejecta morphology of novae can be considered as an important test bench for studying the shaping of many kinds of nebulae. V1280 Sco is one of the slowest dust-forming nova ever historically observed that has experienced a particularly long common-envelope phase. Aims. We performed multi-epoch high-spatial resolution observations of the circumstellar dusty environment of V1280 Sco to investigate the level of asymmetry of the ejecta. Methods. We observed V1280 Sco in 2009, 2010 and 2011 (from t = 877 days after discovery until t = 1664 d) using unprecedented high angular resolution techniques. We used the NACO/VLT adaptive optics system in the J, H and K bands, together with contemporaneous VISIR/VLT mid-IR imaging that resolved the dust envelope of V1280 Sco, and SINFONI/VLT observations secured in 2011. Results. We report the discovery of a dusty hourglass-shaped bipolar nebula. The apparent size of the nebula increased from 0.30 × 0.17 in July 2009 to 0.64 × 0.42 in July 2011. The aspect ratio suggests that the source is seen at high inclination. The central source shines efficiently in the K band and represents more than 56 ± 5% of the total flux in 2009, and 87 ± 6% in 2011. A mean expansion rate of 0.39 ± 0.03 milliarcsec per day is inferred from the VISIR observations in the direction of the major axis, which represents a projected upper limit. Assuming that the dust shell expands in that direction as fast as the low-excitation slow ejecta detected in spectroscopy, this yields a lower limit distance to V1280 Sco of ∼1 kpc; however, the systematic errors remain large due to the complex shape and velocity field of the dusty ejecta. The dust seems to reside essentially in the polar caps and no infrared flux is detected in the equatorial regions in the latest dataset. This may imply that the mass-loss was dominantly polar. Conclusions. V1280 Sco is an excellent test case for studying the temporal evolution of dusty bipolar ejecta. As the nebula expands, observations will be easier and we advocate a yearly monitoring of the source using high angular resolution techniques.