Ingestion Configuration

Dataset Preparation

Dataset Metadata is required to accompany the dataset for it to be recognised by Data Cube. It defines critical metadata of the dataset such as:

  • measurements
  • platform and sensor names
  • geospatial extents and projection
  • acquisition time

Note

Some metadata requires cleanup before they are ready to be loaded.

For more information see Dataset Metadata.

Storage Types

Storage Type is a document that defines the way an input dataset is stored inside the Data Cube.

It controls things like:

  • which measurements are stored
  • what projection the data is stored in
  • what resolution the data is stored in
  • how data is tiled
  • where the data is stored

Multiple storage type definitions can be used to ingest datasets into different projections, resolutions, etc.

For more information see Storage Type.

datacube-config can be used to add storage types. Sample configs are in docs/config_samples.

datacube-config storage add docs/config_samples/ga_landsat_7/ls7_albers.yaml

Note

You should refer to platform in your metadata file to determine which kind of mapping to configure. For example, LANDSAT_5 means you should configure for the Landsat 5 configuration.

Configuration samples are available as part of the open source Github repoistory.

Ingestion

datacube-ingest can be used to ingest prepared datasets:

datacube-ingest -v ingest packages/nbar/LS8_OLITIRS_TNBAR_P54_GALPGS01-002_112_079_20140126 packages/pq/LS8_OLITIRS_PQ_P55_GAPQ01-002_112_079_20140126