During the process of seabed terrain exploration using a multi-beam echo system, it is inevitable to obtain a sounding set containing anomalous points. Conventional methods for eliminating outliers are unable to reduce the disruption caused by outliers over the whole dataset. Furthermore, incomplete consideration is given to the terrain complexity, error magnitude, and outlier distribution. In order to achieve both a high-precision terrain quality estimate and quick detection of depth anomalies, this study suggests a dual robust technique. Firstly, a robust polyhedral function is utilized to solve anomaly detection for large errors. Secondly, the robust kriging algorithm is used for refined outlier removal. Ultimately, the process of dual detection and anomaly removal is achieved. The experimental results demonstrate that DRS technology has the most favorable mean square error and error fluctuation range in the test set, with values of 0.8321 and [−2.0582, 1.9209], respectively, when compared to RPF, WT, GF, and WLS-SVM schemes. Furthermore, DRS is able to adjust to various terrain complexities, discrete distribution features, and cluster outlier detection, as shown by objective indicators and visual outcome maps, guaranteeing a high-quality seabed terrain estimate.