Australian Geoscience Data Cube
 |
The Australian Geoscience Data Cube (AGDC) is an approach to storing, processing and analyzing large collections of earth observation data. The technology is designed to meet challenges of national interest by being agile and flexible with vast amounts of layered grid data.
The AGDC reduces processing time of traditional image analysis by calibrating, pre-computing known extents, pixel alignment and storing metadata in a cell lattice structure. The temporal-pixel aligned data can often be analysed faster across space and time dimensions than previous scene based techniques. This allows the AGDC to be flexible in tackling future challenges and improve analysis times on every-increasing data repositories of earth observation.
The AGDC has also been used internationally [1] to allow countries to maintain ecologically sustainable programs and reduce the difficulty curve of utilizing Remote Sensing data.[2]
Background
The AGDC was originally conceived by Geoscience Australia but is now maintained in a partnership between Geoscience Australia, Commonwealth Scientific and Industrial Research Organisation (CSIRO) and National Computational Infrastructure National Facility (Australia) (NCI). This is made possible by the funding from the partnership and a number of organisations such as National Collaborative Research Infrastructure Strategy (NCRIS).[3]
Analysis ready data, ingestion and indexing
The data processed in the cube is made analysis ready [4] before being ingested and indexed into the AGDC. Analysis ready data is pre-processed data that has applied corrections for instrument calibration (gains and offsets), geolocation (spatial alignment) and radiometry (solar illumination, incidence angle, topography, atmospheric interference). The ingestion process manages the translation of datasets into the storage units while maintaining a database index. The data within the storage and index can be accessed via API calls often compiled within code such as Python (programming language).
Example:
s2a_l1c = dc.load(product='s2a_level1c_granule',x=(147.36, 147.41), y=(-35.1, -35.15), measurements=['04','03','02'], output_crs='EPSG:4326', resolution=(-0.00025,0.00025))
Datasets currently stored
- Geoscience Australia Landsat Surface Reflectance (1987 to present)
- Landsat Pixel Quality
- Landsat Fractional Cover
- Landsat NDVI
Datasets that have been piloted
- USGS Landsat Surface Reflectance
- SRTM DEM
- Himawari 8
- MODIS
- Sentinel-2 L1C / S2A
- Australian Gridded Climate Data
Open source
The AGDC code base is situated in GitHub as an open repository.[5] The core code base moved to the Open Data Cube in early 2017 as part of an international collaboration. Whilst the code base is the Open Data Cube, individual cubes exist as their own right such as the AGDC on the National Computational Infrastructure National Facility (Australia) (NCI) using the High-Performance Computing Cluster HPCC. The core code can be installed on personal computers or public computers (using git) and has many unit tests.
Documentation for the code base exists on Read the Docs.[6]
Challenges of the AGDC
The AGDC is designed to meet nationally significant challenges such as the following.
- Sustainability
- Environment
- Water resource management
- Disaster assist
- Policy development
- Community planning
- Forest preservation
- Carbon measurement
International awards
The AGDC won the 2016 Content Platform of the Year award from Geospatial World Forum.[7][8]
References
- ^ "CEOS". Archived from the original on 2019-11-14. Retrieved 2017-03-14.
- ^ "CEOS Data Cube Platform version 2 (CEOS2)". software.nasa.gov. Archived from the original on 2017-03-15.
- ^ "National Collaborative Research Infrastructure Strategy (NCRIS)". 5 October 2021.
- ^ "Open Data Cube | Open Source".
- ^ "Opendatacube/Datacube-core". GitHub. 20 January 2022.
- ^ "Open Data Cube Manual — Open Data Cube 1.8 documentation".
- ^ "Australian Geoscience Data Cube innovation recognised on world stage". 23 January 2017.
- ^ http://geospatialworldforum.org/