The Philippine Sea plate subducts beneath the Eurasia plate at the Nankai Trough, northwestern Pacific, causing crustal deformation, mega‐thrust earthquakes, and tsunami events. Shikoku Island, 150 km northwest of the trough, experiences both coseismic and interseismic deformation. Coastal sediments potentially record vertical crustal movements as relative sea level (RSL) changes. We studied sedimentary facies and microfossil ostracodes in core SKM from southwestern Shikoku Island for evidence of middle Holocene tsunami events and deformation. The core sediments included nine event layers corresponding to storm or tsunami events. Using modern analog techniques, we estimated RSLs from the ostracode assemblages of core SKM and 13 other cores from Shikoku Island and the surrounding region. Then, we subtracted RSL changes due to glacio‐hydro isostatic adjustment from the estimated RSLs to estimate vertical tectonic movement rates in these cores between 8.6 and 4.7 ka. The inferred RSL changes suggest that the Sukumo site has experienced both uplift and subsidence since 8.6 ka. Before 6.6 ka, rates of tectonic crustal movement were higher than the modern‐day rate, and its spatial distribution also differed. After 6.6 ka, tectonic crustal movement showed a similar spatial pattern and occurred at rates close to the modern day interseismic rate. The spatial pattern and rates of tectonic crustal movement could be caused by changes in rupture areas between the Eurasia and the Philippine Sea plates beneath the Shikoku Island and in stress condition of the asthenosphere. Some of the vertical displacements can be explained by the movements of local active faults.