The determination of the land geoid and the marine geoid involves different data sets and calculation strategies. It is a hot issue at present to construct the unified land–ocean quasi-geoid by fusing multi-source data in coastal areas, which is of great significance to the construction of land–ocean integration. Classical geoid integral algorithms such as the Stokes theory find it difficult to deal with heterogeneous gravity signals, so scholars have gradually begun using radial basis functions (RBFs) to fuse multi-source data. This article designs a multi-layer RBF network to construct the unified land–ocean quasi-geoid fusing measured terrestrial, shipborne, satellite altimetry and airborne gravity data based on the Remove–Compute–Restore (RCR) technique. EIGEN-6C4 of degree 2190 is used as a reference gravity field. Several core problems in the process of RBF modeling are studied in depth: (1) the behavior of RBFs in the spatial domain; (2) the locations of RBFs; (3) ill-conditioned problems of the design matrix; (4) the effect of terrain masses. The local quasi-geoid with a 1′ resolution is calculated, respectively, on the flat east coast and the rugged west coast of the United States. The results show that the accuracy of the quasi-geoid computed by fusing four types of gravity data in the east coast experimental area is 1.9 cm inland and 1.3 cm on coast after internal verification (the standard deviation of the quasi-geoid w.r.t GPS/leveling data). The accuracy of the quasi-geoid calculated in the west coast experimental area is 2.2 cm inland and 2.1 cm on coast. The results indicate that using RBFs to calculate the unified land–ocean quasi-geoid from heterogeneous data sets has important application value.