Tradeoffs among the spatial, spectral, and temporal resolutions of satellite sensors make it difficult to acquire remote sensing images at both high spatial and high temporal resolutions from an individual sensor. Studies have developed methods to fuse spatiotemporal data from different satellite sensors, and these methods often assume linear changes in surface reflectance across time and adopt empirical rules and handcrafted features. Here we propose a dense spatiotemporal fusion (DenseSTF) network based on the convolutional neural network to deal with these problems. DenseSTF uses a patch-to-pixel modeling strategy which can provide abundant texture details for each pixel in the target fine image to handle heterogeneous landscapes, and models both forward and backward temporal dependence to account for land cover changes. Moreover, DenseSTF adopts a mapping function with little assumptions and empirical rules which allows for establishing reliable relationships between the coarse and fine images. We tested DenseSTF in three contrast scenes with different degrees of heterogeneity and temporal changes, and made comparisons with three rule-based fusion approaches and three convolutional neural networks. Experimental results indicate that DenseSTF can provide accurate fusion results and outperform the other tested methods, especially when the land cover changes abruptly. The structure of the deep learning networks largely impacts the success of data fusion. Our study developed a novel approach based on the convolutional neural network using a patch-to-pixel mapping strategy and highlighted the effectiveness of the deep learning networks in spatiotemporal fusion of the remote sensing data.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.