Autocorrelation functions (ACFs) for ambient seismic noise are considered to be useful tools for estimating temporal changes in the subsurface structure. Velocity changes at Hakone volcano in central Japan, where remarkable swarm activity has often been observed, were investigated in this study. Significant velocity changes were detected during two seismic activities in 2011 and 2013. The 2011 activity began immediately after the 2011 Tohoku-oki earthquake, suggesting remote triggering by the dynamic stress changes resulting from the earthquake. During the 2013 activity, which exhibited swarm-like features, crustal deformations were detected by Global Navigation Satellite System (GNSS) stations and tiltmeters, suggesting a pressure increment of a Mogi point source at a depth of 7 km and two shallow open cracks. Waveforms that were bandpass-filtered between 1 and 3 Hz were used to calculate ACFs using a one-bit correlation technique. Fluctuations in the velocity structure were obtained using the stretching method. A gradual decrease in the velocity structure was observed prior to the 2013 activity at the KOM station near the central cone of the caldera, which started after the onset of crustal expansion observed by the GNSS stations. Additionally, a sudden significant velocity decrease was observed at the OWD station near a fumarolic area just after the onset of the 2013 activity and the tilt changes. The changes in the stress and strain caused by the deformation sources were likely the main contributors to these decreases in velocity. The precursory velocity reduction at the KOM station likely resulted from the inflation of the deep Mogi source, whereas the sudden velocity decrease at the OWD station may reflect changes in the strain caused by the shallow open-crack source. Rapid velocity decreases were also detected at many stations in and around the volcano after the 2011 Tohoku-oki earthquake. The velocity changes may reflect the redistribution of hydrothermal fluid in response to the large stress perturbation caused by the 2011 Tohoku-oki earthquake.