Scour monitoring is an important measurement process to determine the soil erosion level at the pillar of bridge. Image-based approach is attractive because it allows monitoring process to be conducted continuously without halting the flow of the water during experiment. Scour images provide abundance of information from a single source of camera sensor. However, this information may appear in different features, orientation, size and brightness. Therefore, it is important to detect and recognise features that are related to scour monitoring and filtered out irrelevant features like image noises and artefacts. This paper presents implementation of image processing techniques to extract various information from scour images. Image inpainting technique is used to separate information of scour level and scale markers into two different images. A proposed gradient of marginal histogram technique is used to detect the horizontal scale line markers and scour level. Backpropagation neural network is used to recognise scale text markers and convert the measurement of the scour level from a pixel unit into a centimetre unit. Interpolation techniques are used to connect scour points and delineate the boundary indicating the level of the scour. Time series acquisition of scour images allows observation of the temporal variation of the scour levels. Results show that the proposed approach achieved higher accuracy than existence method. This approach allows the detection of the scour level for even and uneven sediments, contributing to the high accurate results at the spatial and temporal measurements, thus potentially offering continuous scour monitoring solution.