Gravity measurement is a basic geophysical method for non-destructive exploration of mineral resources and hydrocarbons and also for geological studies. The quality of gravity data measured is constantly degraded by a low signal-to-noise ratio. Noise attenuation thus plays a vital role in processing and interpreting gravity field data. The non-local means (NLM) filtering was first and successfully introduced to attenuate randomly distributed noise situated in seismic records in geophysical community. However, less attention has been drawn to apply NLM to denoise potential field data, since the success of NLM is guaranteed by carefully tuned parameters and large computational costs. Here we propose the modified NLM (MNLM), based on unweighted Euclidean distance and integral image method, to denoise gravity datasets. The unweighted Euclidean distance is used to reduce the uncertainty and complexity of tuning the control parameters involved, and the integral image strategy is applied to avoid the enormous computational cost of the NLM. Since these filtering properties are desirable to mitigate noise in gravity datasets, we test and evaluate the MNLM filter on noisy synthetic gravity models with uniform and normal distributions and on real data from the Mariana trench and Slovakia, and compare it to other standard and representative denoising filters. The results on synthetic and field datasets confirm the high speed of this modified algorithm and show that, it removes noise most effectively while clearly preserving and not blurring structural details with less tuned parameters. The MNLM filter can therefore be considered as a promising and novel algorithm for denoising gravity data.