Open Data Cube Manual

The Open Data Cube provides an integrated gridded data analysis environment for decades of analysis ready Earth observation satellite and related data from multiple sources.

• About & Core Concepts
  • Overview
  • Use Cases
    • Overview & Use Cases
    • Architectural Overview
    • Datasets
    • Types of Datasets
      • Indexed Datasets
      • Ingested Datasets
    • Products
      • Product Definition
        • Example Product Definition
        • Product Definition API
    • Metadata Types
    • Extensions & Related Software
      • Open Web Services
      • Data Cube Explorer
      • Cube in a Box
      • Data Cube Alchemist
      • Data Cube Statistician
      • Data Cube Stats
    • Existing Deployments
      • Digital Earth Australia
      • Digital Earth Africa
      • Swiss Data Cube
      • Vietnam Open Data Cube
      • INEGI - Mexico Open Data Cube
      • Brazil Data Cube
      • Armenian Data Cube
      • Mongolian and Kyrgyz Data Cubes
• Data Access & Analysis
  • Connecting to the Open Datacube
    • Minimal Example
    • API
  • Product Discovery
    • Product Listings
    • Measurement Listings
  • Dataset Searching & Querying
    • Finding Datasets
    • Inspecting Datasets
  • Data Loading
    • Loading Datasets
    • Querying and Loading
  • Virtual Products
    • Introduction
    • Using virtual products
    • Design
    • Components
      • Product (Input)
      • Collate (Combining)
      • Juxtapose (Combining)
      • Reproject (Data modifying)
      • Transform (Data modifing)
      • Aggregate (Summary statistics)
    • Built in Transforms
      • Expressions
      • Make mask
      • Apply mask
      • To Float
      • Rename
      • Select
    • User-defined transformations
  • Data Loading in Depth
    • Types of Data Loading
      • Ad hoc data access
      • Large scale processing data access
    • Data Structures
      • Virtual Storage Resource
    • Data load in detail
      • Internal interfaces
    • Fuse function customisation
      • Problems with the current approach to fusing
    • Lazy load with dask
    • Limitations and problems
      • Possible Solutions
    • Storage Drivers
      • GDAL
      • S3IO
    • Supporting Diagrams
      • Data Read Process
      • Storage Classes
  • Using Juypter Notebooks
    • Digital Earth Australia Notebooks
    • Digital Earth Africa Notebooks
    • DEA and DE Africa Tools code
• Installing and managing the Open Data Cube
  • Mac OSX Developer Setup
    • Required software
    • Python and packages
      • Anaconda Python
    • Postgres database configuration
    • Open Data Cube source and development configuration
    • Verify it all works
  • Ubuntu Developer Setup
    • Required software
    • Python and packages
      • Anaconda Python
    • Postgres database configuration
    • Open Data Cube source and development configuration
    • Verify it all works
  • Windows Developer Setup
    • Required software
    • Python and packages
      • Anaconda Python
    • Postgres database configuration
    • Open Data Cube source and development configuration
    • Verify it all works
  • Command Line Tools
    • datacube
      • dataset
        • add
        • archive
        • info
        • purge
        • restore
        • search
        • update
        • uri-search
      • ingest
      • metadata
        • add
        • list
        • show
        • update
      • product
        • add
        • list
        • show
        • update
      • system
        • check
        • init
      • user
        • create
        • delete
        • grant
        • list
    • datacube-search
      • datasets
      • product-counts
    • datacube-worker
  • Database Setup
    • Install PostgreSQL
    • Create Database
    • Create Configuration File
    • Initialise the Database Schema
      • datacube system
  • Metadata Types
  • Product Definitions
  • Dataset Documents
    • EO3 Format
      • 3D dataset metadata
    • EO (deprecated)
  • Step-by-step Guide to Indexing Data
    • Introduction
    • Steps Overview
    • Step 1. Creating a Product Definition
      • Loading Product Definitions
    • Step 2. Ensure Dataset Documents are complete
    • Step 3. Run the Indexing process
  • Indexing data from Amazon (AWS S3)
    • Configuring AWS CLI Credentials
    • Install AWS S3 Indexing Scripts
    • S3 Indexing Example
  • Extending the Open Data Cube
    • Support for Plug-in drivers
    • Data Read Plug-ins
      • Example code to implement a reader driver
      • Example Pickle Based Driver
    • 3D Data Read Plug-ins
      • Example code to implement a 3D read
      • Example Xarray Based 3D Driver
    • Data Write Plug-ins
      • Example code to implement a writer driver
      • NetCDF Writer Driver
    • Index Plug-ins
      • Default Implementation
      • Null Implementation
      • Memory Implementation
      • PostGIS Implementation
    • Drivers Plugin Management Module
    • Data Cube Drivers API
    • References and History
  • Data Preperation Scripts
  • Landsat Samples
  • 1. Preparing USGS Landsat Collection 1 - LEVEL1
  • 2. Preparing USGS Landsat Surface Reflectance - LEDAPS
  • 3. Indexing data on AWS, an example using Sentinel-2
  • Custom Prepare Scripts
  • Ingesting Data
    • Ingestion Configuration
    • Ingestion Config
    • Ingest Some Data
      • datacube ingest
• API Reference
  • Datacube Class
  • Dataset
  • Dataset Type
  • Geometry Classes
    • datacube.utils.geometry.Coordinate
    • datacube.utils.geometry.BoundingBox
    • datacube.utils.geometry.CRS
    • datacube.utils.geometry.Geometry
    • datacube.utils.geometry.GeoBox
    • datacube.utils.geometry.BoundingBox
    • datacube.utils.geometry.gbox.GeoboxTiles
    • datacube.model.GridSpec
    • datacube.utils.geometry.CRSError
    • datacube.utils.geometry.CRSMismatchError
  • Measurement
  • Metadata Type
  • Range
  • Query
  • Tile
  • Dataset & Product Discovery
    • datacube.Datacube.list_products
    • datacube.Datacube.list_measurements
  • Dataset Querying
    • datacube.index.abstract.AbstractDatasetResource.get
    • datacube.index.abstract.AbstractDatasetResource.search
    • datacube.index.abstract.AbstractDatasetResource.search_by_metadata
    • datacube.index.abstract.AbstractDatasetResource.search_by_product
    • datacube.index.abstract.AbstractDatasetResource.search_eager
    • datacube.index.abstract.AbstractDatasetResource.search_product_duplicates
    • datacube.index.abstract.AbstractDatasetResource.search_returning
    • datacube.index.abstract.AbstractDatasetResource.search_summaries
    • datacube.index.abstract.AbstractDatasetResource.has
    • datacube.index.abstract.AbstractDatasetResource.bulk_has
    • datacube.index.abstract.AbstractDatasetResource.can_update
    • datacube.index.abstract.AbstractDatasetResource.count
    • datacube.index.abstract.AbstractDatasetResource.count_by_product
    • datacube.index.abstract.AbstractDatasetResource.count_by_product_through_time
    • datacube.index.abstract.AbstractDatasetResource.count_product_through_time
    • datacube.index.abstract.AbstractDatasetResource.get_derived
    • datacube.index.abstract.AbstractDatasetResource.get_field_names
    • datacube.index.abstract.AbstractDatasetResource.get_locations
    • datacube.index.abstract.AbstractDatasetResource.get_archived_locations
    • datacube.index.abstract.AbstractDatasetResource.get_datasets_for_location
  • Product Querying
    • datacube.index.abstract.AbstractProductResource.can_update
    • datacube.index.abstract.AbstractProductResource.get
    • datacube.index.abstract.AbstractProductResource.get_by_name
    • datacube.index.abstract.AbstractProductResource.get_with_fields
    • datacube.index.abstract.AbstractProductResource.search
    • datacube.index.abstract.AbstractProductResource.search_robust
    • datacube.index.abstract.AbstractProductResource.get_all
  • Data Loading
    • datacube.Datacube.load
    • Internal Loading Functions
      • datacube.Datacube.find_datasets
      • datacube.Datacube.group_datasets
      • datacube.Datacube.load_data
    • Loading 3D Datasets
  • Masking
  • Bit Masking
    • Masking No Data Values
    • Masking with Bit-Flag Measurements
      • How to Define Meanings on Measurements
      • How to Create Masks within code
  • Dataset Writing
    • datacube.index.abstract.AbstractDatasetResource.add
    • datacube.index.abstract.AbstractDatasetResource.add_location
    • datacube.index.abstract.AbstractDatasetResource.archive
    • datacube.index.abstract.AbstractDatasetResource.archive_location
    • datacube.index.abstract.AbstractDatasetResource.remove_location
    • datacube.index.abstract.AbstractDatasetResource.restore
    • datacube.index.abstract.AbstractDatasetResource.restore_location
    • datacube.index.abstract.AbstractDatasetResource.update
  • Product Writing
    • datacube.index.abstract.AbstractProductResource.from_doc
    • datacube.index.abstract.AbstractProductResource.add
    • datacube.index.abstract.AbstractProductResource.update
    • datacube.index.abstract.AbstractProductResource.update_document
    • datacube.index.abstract.AbstractProductResource.add_document
  • Creating Geometries
    • datacube.utils.geometry.point
    • datacube.utils.geometry.multipoint
    • datacube.utils.geometry.line
    • datacube.utils.geometry.multiline
    • datacube.utils.geometry.polygon
    • datacube.utils.geometry.multipolygon
    • datacube.utils.geometry.multigeom
    • datacube.utils.geometry.box
    • datacube.utils.geometry.sides
    • datacube.utils.geometry.polygon_from_transform
  • Spatial Operations
    • datacube.utils.geometry.unary_union
    • datacube.utils.geometry.unary_intersection
    • datacube.utils.geometry.bbox_union
    • datacube.utils.geometry.bbox_intersection
    • datacube.utils.geometry.lonlat_bounds
    • datacube.utils.geometry.projected_lon
    • datacube.utils.geometry.clip_lon180
    • datacube.utils.geometry.chop_along_antimeridian
  • Utilities
    • datacube.utils.geometry.assign_crs
    • datacube.utils.geometry.crs_units_per_degree
    • datacube.utils.geometry.geobox_union_conservative
    • datacube.utils.geometry.geobox_intersection_conservative
    • datacube.utils.geometry.scaled_down_geobox
    • datacube.utils.geometry.intersects
    • datacube.utils.geometry.common_crs
    • datacube.utils.geometry.is_affine_st
    • datacube.utils.geometry.apply_affine
    • datacube.utils.geometry.roi_boundary
    • datacube.utils.geometry.roi_is_empty
    • datacube.utils.geometry.roi_is_full
    • datacube.utils.geometry.roi_intersect
    • datacube.utils.geometry.roi_shape
    • datacube.utils.geometry.roi_normalise
    • datacube.utils.geometry.roi_from_points
    • datacube.utils.geometry.roi_center
    • datacube.utils.geometry.roi_pad
    • datacube.utils.geometry.scaled_down_shape
    • datacube.utils.geometry.scaled_down_roi
    • datacube.utils.geometry.scaled_up_roi
    • datacube.utils.geometry.decompose_rws
    • datacube.utils.geometry.affine_from_pts
    • datacube.utils.geometry.get_scale_at_point
    • datacube.utils.geometry.native_pix_transform
    • datacube.utils.geometry.compute_reproject_roi
    • datacube.utils.geometry.split_translation
    • datacube.utils.geometry.compute_axis_overlap
    • datacube.utils.geometry.w_
    • datacube.utils.geometry.warp_affine
    • datacube.utils.geometry.rio_reproject
  • Amazon Web Services (AWS)
    • datacube.utils.aws.s3_client
    • datacube.utils.aws.s3_open
    • datacube.utils.aws.s3_head_object
    • datacube.utils.aws.s3_fetch
    • datacube.utils.aws.s3_dump
    • datacube.utils.aws.s3_url_parse
    • datacube.utils.aws.auto_find_region
    • datacube.utils.aws.get_aws_settings
    • datacube.utils.aws.get_creds_with_retry
    • datacube.utils.aws.mk_boto_session
    • datacube.utils.aws.ec2_current_region
    • datacube.utils.aws.ec2_metadata
    • datacube.utils.aws.configure_s3_access
  • Writing GeoTIFFs
    • datacube.utils.cog.write_cog
    • datacube.utils.cog.to_cog
  • Dask
    • datacube.utils.dask.start_local_dask
    • datacube.utils.dask.partition_map
    • datacube.utils.dask.pmap
    • datacube.utils.dask.compute_tasks
    • datacube.utils.dask.save_blob_to_file
    • datacube.utils.dask.save_blob_to_s3
  • Grid Workflow