Remotely sensed images captured from a camera mounted on a UAV (unmanned aerial vehicle) are exposed to noise caused by internal factors, such as the UAV system itself or external factors such as atmospheric conditions. Such images need to be restored before they can undergo further processing stages. This study aims to analyse the effects of salt and pepper noise on a UAV image and restore the image by removing the noise effects. In doing so, a UAV image, with red, green and blue channel and containing regions of different spectral properties, is experimented with salt and pepper noise of different densities. Image restoration procedure is formulated using median filtering of variable sizes. Peak-signal to noise ratio (PSNR) and mean square error (MSE) analysis are performed to measure image quality before and after restoration. An optimal filter size is chosen based on the highest PSNR of the restored image. The results show that the effects of noise on UAV images are dependent on the spectral properties of the image channels and the regions of interest. The proposed restoration works best for images with low-compared to high-density noises. Blue channel is found having the largest variation of optimal filter size, 18.5, compared to other channels because of the high response to noise within its short spectral wavelength region. Landscape's vegetation has the largest variation of optimal filter size, 22, compared to other regions due to the sensitivity of its dark spectral properties.