The South China Sea (SCS) is the result of several major tectonic plates’ interaction, and a more detailed density structure is essential to study the tectonic evolution of the marginal seas. In the past decades, density studies in the SCS have focused more on profiles, and the lateral density structure has been less mentioned by scholars. Previously, gravity data have been used many times to study the SCS; this can satisfy the demands of large-scale lateral studies, but a point that cannot be ignored is that the gravity method has certain deficiencies in vertical resolution. Therefore, this paper applies a gravity multiscale analysis method to isolate the Bouguer gravity anomaly signal in the SCS, and then uses the power spectrum method to estimate the mean field source depth, which corresponds to the decomposed gravity anomaly signal. On this basis, we modeled stratified crustal and upper mantle structures in the SCS to determine the lateral density distribution at different depths. The results show the following: (1) There are two large high-density materials in the SCS. The first one is located near the Yinggehai Basin, where the depth ranges from 42.4 km to 71.2 km, with a diameter of nearly 220 km; the second one is located near the northwest sub-basin, where the depth ranges from 106.8 km to 128.8 km, which is probably part of the Paleo-Pacific remnant in the SCS. (2) The proto South China Sea subduction zone is located in the northwestern part of the Sulu Sea and has a northeast–southwest trend.