TWINNS: The Copernicus programme developed by the European Space Agency provides, with missions such as Sentinel-1 (S1) and Sentinel-2 (S2), radar and optical (multi-spectral) imagery, respectively, at 10~m spatial resolution with revisit time around 5 days. Such high temporal resolution allows to collect Satellite Image Time Series (SITS) that support a plethora of Earth surface monitoring tasks. How to effectively combine the complementary information provided by such sensors remains an open problem in the remote sensing field. In this work, we propose a deep learning architecture to combine information coming from S1 and S2 time series, namely TWINNS (TWIn Neural Networks for Sentinel data), able to discover spatial and temporal dependencies in both types of SITS. The proposed architecture is devised to boost the land cover classification task by leveraging two levels of complementarity, i.e., the interplay between radar and optical SITS as well as the synergy between spatial and temporal dependencies.
Article: Link (Interactive Map HERE)
Keywords: Satellite Image Time Series, Sentinel-1, Sentinel-2 – Recurrent Neural network, Convolutional Neural Network. Acknowledgments:#DigitAg, GeoSud/Theia
OD2RNN: Radar and Optical Satellite Image Time Series (SITS) are sources of information that are commonly employed to monitor earth surfaces for tasks related to ecology, agriculture, mobility, land management planning and land cover monitoring. Many studies have been conducted using one of the two sources, but how to smartly combine the
complementary information provided by radar and optical SITS is still an open challenge. In this context, we propose a new neural architecture for the combination of Sentinel-1 (S1) and Sentinel- 2 (S2) imagery at object level, applied to a real-world land cover classification task. Experiments carried out on the Reunion Island, a overseas department of France in the Indian Ocean, demonstrate the significance of our proposal.
(Interactive Map HERE)
Keywords: Satellite Image Time Series, Sentinel-1, Sentinel-2 – Recurrent Neural network, Two-Branch Architecture. Acknowledgments:GeoSud/Theia
MultiResoLCC: Nowadays, Earth Observation systems provide a multitude of heterogeneous remote sensing data. How to manage such richness leveraging its complementarity is a crucial challenge in modern remote sensing analysis. Considering optical Very High Spatial Resolution (VHSR) images, satellites obtain both Multi Spectral (MS) and panchromatic (PAN) images at different spatial resolutions. VHSR images are extensively exploited to produce land cover maps to deal withagricultural, ecological, and socioeconomic issues as well as assessing ecosystem status, monitoring biodiversity and providing inputs to conceive food risk monitoring systems. Common ways to produce land cover maps from such VHSR images typically opt for a prior pansharpening of the multi-resolution sources for a full resolution processing. Here, we propose a new deep learning architecture to jointly use PAN and MS imagery for a direct classification without any prior image fusion or resampling process. By managing the spectral information at its native spatial resolution, our method, named MultiResoLCC, aims at avoiding the possible information bias induced by pansharpening or any other hand-crafted preprocessing.
(Interactive Map HERE)
Keywords: VHSR imagery, Data Fusion – Convolutional Neural Networks, Two-Branch Architecture. Acknowledgments:GeoSud/Theia, DigitAg
DUPLO: Nowadays, modern Earth Observation systems continuously generate huge amounts of data. A notable example is represented by the Sentinel-2 mission, which provides images at high spatial resolution (up to 10m) with high temporal revisit period (every 5 days), which can be organized in Satellite Image Time Series (SITS). While the use of SITS has been proved to be beneficial in the context of Land Use/Land Cover (LULC) map generation, unfortunately, machine learning approaches commonly leveraged in remote sensing field fail to take advantage of spatio-temporal dependencies present in such data. Recently, new generation deep learning methods allowed to significantly advance research in this field. These approaches have generally focused on a single type of neural network, i.e., Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs), which model different but complementary information: spatial autocorrelation (CNNs) and temporal dependencies (RNNs). In this work, we propose the first deep learning architecture for the analysis of optical Sentinel-2 SITS data, namely DUPLO (DUal view Point deep Learning architecture for time series classificatiOn), that combines Convolutional and Recurrent neural networks to exploit their complementarity. Our hypothesis is that, since CNNs and RNNs capture different aspects of the data, a combination of both models would produce a more diverse and complete representation of the information for the underlying land cover classification task.
(Interactive Map HERE)
Keywords: , Sentinel2, Satellite Images Time Series – Recurrent Neural Networks, Attention, Convolutional Neural Networks, Auxiliary Classifiers. Acknowledgments:GeoSud/Theia, DigitAg, Grant n° PNTS-2018-5
Multi-Source data exploitation for Land Cover Classification
M3Fusion: Modern Earth Observation (EO) systems provide remote sensing data a different temporal and spatial resolutions. Among all the available spatial mission, today the Sentinel-2 program supplies high temporal (every 5 days) and high spatial resolution (10m) images that can be useful to monitor land cover dynamics. On the other hand, Very HighSpatial Resolution (VHSR) imagery is still essential to figure out land cover mapping characterized by fine spatial patterns. Understanding how to jointly leverage these complementary sources in an efficient way when dealing with land cover mapping is acurrent challenge in remote sensing. With the aim of providing land cover mapping through the fusion of multi-temporal High Spatial Resolution and VHSR satellite images, we propose a suitable end-to-end Deep Learning framework, namely M3Fusion, which is able to simultaneously leverage the temporal knowledge contained in time series data as well as the fine spatial information availablein VHSR images.
(Interactive Map HERE)
Keywords: , Sentinel2, VHSR imagery, Satellite Images Time Series – Recurrent Neural Networks, Attention, Convolutional Neural Networks, Auxiliary Classifiers. Acknowledgments:GeoSud/Theia, DigitAg