Abstract
AIS data from ships is excellent for analyzing single-ship movements and monitoring all ships within a specific area. However, the AIS data needs to be cleaned, processed, and stored before being usable. This paper presents a system consisting of an efficient and modular ETL process for loading AIS data, as well as a distributed spatial data warehouse storing the trajectories of ships. To efficiently analyze a large set of ships, a raster approach to querying the AIS data is proposed. A spatially partitioned data warehouse with a granularized cell representation and heatmap presentation is designed, developed, and evaluated. Currently the data warehouse stores 312 million kilometers of ship trajectories and more than 8 billion rows in the largest table. It is found that searching the cell representation is faster than searching the trajectory representation. Further, we show that the spatially divided shards enable a consistently good scale-up for both cell and heatmap analytics in large areas, ranging between 354% to 1164% with a 5x increase in workers
Original language | English |
---|---|
Title of host publication | IEEE International Conference on Mobile Data Management |
Publisher | IEEE (Institute of Electrical and Electronics Engineers) |
Publication date | 2024 |
Pages | 211-218 |
ISBN (Electronic) | 979-8-3503-7455-1 |
DOIs | |
Publication status | Published - 2024 |
Event | 2024 25th IEEE International Conference on Mobile Data Management (MDM): MDM - Brussels, Belgium Duration: 24 Jul 2024 → 27 Jul 2024 |
Conference
Conference | 2024 25th IEEE International Conference on Mobile Data Management (MDM) |
---|---|
Country/Territory | Belgium |
City | Brussels |
Period | 24/07/2024 → 27/07/2024 |