Adopting deep learning methods for airborne RGB fluvial scene classification.

Abstract

River environments are among the world’s most threatened ecosystems. Enabled by the rapid development of drone technology, hyperspatial resolution (<10 cm) images of fluvial scenes are now a common data source used to better understand these sensitive habitats. However, the task of image classification remains challenging for this type of imagery and the application of traditional classification algorithms such as maximum likelihood, still in common use among the river remote sensing community, yields unsatisfactory results. We explore the possibility that a classifier of river imagery based on deep learning methods can provide a significant improvement in our ability to classify fluvial scenes. We assemble a dataset composed of existing RGB images from 11 rivers in Canada, Italy, Japan, the UK, and Costa Rica. The images were labelled into 5 classes. In total, >5 billion pixels were labelled and partitioned for the tasks of training (1 billion pixels) and validation (4 billion pixels). We develop a novel supervised learning workflow based on the NASNet convolutional neural network (CNN) called ‘CNN-Supervised Classification’ (CSC). First, we compare the classification success of maximum likelihood, a multilayer perceptron, a random forest, and CSC. Results show F1 scores (a best-practice quality meter in machine learning) of 71%, 78%, 72% and 95%, respectively. Second, we train our classifier using data for 5 of 11 rivers. We then predict the validation data for all 11 rivers. For the 5 rivers that were used in model training, F1 scores reach 98%. For the 6 rivers not used in model training, F1 scores are 90%. We reach two conclusions. First, in the traditional workflow where images are classified one at a time, CSC delivers an unprecedented mix of labour savings and classification F1 scores above 95%. Second, deep learning can determine land-cover classifications (F1 = 90%) for rivers not used in training. This demonstrates the potential to train a generalised open-source deep learning model for airborne river surveys suitable for most rivers ‘out of the box’. Research efforts should now focus on further development of a new generation of deep learning classification tools that will encode human image interpretation abilities and allow for fully automated, potentially real-time, interpretation of riverine landscape images.