The practice of distributed computing framework in spatial big data processing

Authors: Hu Linshu*,
Topics: Geographic Information Science and Systems, Transportation Geography, Land Use
Keywords: Parallel Computation; Spatial Big Data; Spark; Flink
Session Type: Guided Poster
Day: 4/4/2019
Start / End Time: 3:05 PM / 4:45 PM
Room: Roosevelt 3.5, Marriott, Exhibition Level
Presentation File: No File Uploaded

Spatial data analysis and application for big data is an important development direction of Geographical Information System(GIS) in the future, and distributed computing framework is the key solution to support the development. The existing distributed computing frameworks (such as Spark, Flink, etc.) are not well supported by the calculation of spatio data. This paper conducts experimental research on static and real-time spatial data analysis and processing: for static spatial data, based on Spark batch computing framework, through the hybrid of spatial grid index and R-tree index, the intersection algorithm of China's national billion-level land use data is realized. For the online real-time streaming data, taking the real-time traffic congestion data of Shandong Province as an example, based on the Flink flow computing framework, the pyramid tile dynamic construction under multiple concurrency cases is realized by grid index and image resampling. The practical experimental results in spatial big data analysis and processing show that the existing distributed computing framework has great efficiency advantages compared with traditional methods in the analysis of static and real-time spatial computing and processing under the support of big data environment.

Abstract Information

This abstract is already part of a session. View the session here.

To access contact information login