Authors: Zijun Yang*, Geography, University Of Illinois, Urbana Champain, Chunyuan Diao, University of Illinois at Urbana-Champaign
Topics: Remote Sensing
Keywords: spatiotemporal fusion, deep learning, CNN, LSTM, phenological change
Session Type: Paper
Presentation File: No File Uploaded
Spatiotemporal image fusion provides a feasible solution to providing dense time-series remote sensing data with detailed spatial information, which most remote sensing sensors cannot provide but is highly desired in dynamic systems studies. However, previous fusion methods have limitations in predicting fast and/or transient changes (e.g. phenological changes). A systematic approach to assessing and understanding how varying levels of temporal phenological changes affect fusion results is also lacking. Our objective is to develop an innovative hybrid deep learning model that effectively and robustly fuses the satellite imagery of various spatial and temporal resolutions. A systematic image fusion framework that accommodates various levels of temporal phenological changes is devised to assess the model performance and robustness. We propose a novel hybrid deep learning fusion model that integrates super-resolution convolutional neural network (SRCNN), enhancing spatial information, and long short-term memory (LSTM), learning the changing patterns through the time-series images. To systematically assess the effects of phenological changes, we identify major phenological transition dates and design three scenarios representing rapid, moderate, and minimal phenological changes. Results show that the hybrid deep learning model yields significantly better and more robust results in the presence of rapid or moderate phenological changes. The hybrid deep learning model has great potentials to provide dense time-series remote sensing imagery with detailed spatial information for dynamics systems studies. The innovative approach to understanding phenological changes’ effect will help us better comprehend the strengths and weaknesses of current and future fusion models and can be referenced for future studies.