Masking No Data Values¶
Masking with Bit-Flag Measurements¶
One of the common types of data used with the Data Cube contains discrete values stored within a numeric value. These values are often classifications and outputs from tests, and need to be interpreted in specific ways, not as simple scalar values. They are often used as a mask to exclude observations which deemed unsuitable for a given analysis. For example, we want to exclude observations of clouds when we are interested in what is on the ground.
Several methods are used when encoding these types of variables:
On-off bit flags, for a particular binary bit
Collections of bits that can indicate more than two possible values
Looking for a specific value stored using all available bits in the variable
From prior work, it is very easy to make mistakes when using these types of variables, which can lead to processing the wrong set of observations, and also making it quite difficult to read the code using them, and ensuring that they are used in a consistent way in different places.
Open Data Cube provides a way of describing the meanings that can be encoded in variables, which can then be used to give a readable method when using that variable.
How to Define Meanings on Measurements¶
How to Create Masks within code¶
Returns either a Pandas Dataframe (with_pandas=True - default) or a string (with_pandas=False) describing the available flags for a masking variable
Interprets the flags_definition attribute on the provided variable and returns a Pandas Dataframe or string like:
Bits are listed from the MSB (bit 13) to the LSB (bit 0) Bit Value Flag Name Description 13 0 cloud_shadow_fmask Cloud Shadow (Fmask) 12 0 cloud_shadow_acca Cloud Shadow (ACCA) 11 0 cloud_fmask Cloud (Fmask) 10 0 cloud_acca Cloud (ACCA)
variable – Masking xarray.Dataset or xarray.DataArray
Pandas Dataframe or str
Returns a mask array, based on provided flags
When multiple flags are provided, they will be combined in a logical AND fashion.
>>> make_mask(pqa, cloud_acca=False, cloud_fmask=False, land_obs=True)
>>> make_mask(pqa, **GOOD_PIXEL_FLAGS)
where GOOD_PIXEL_FLAGS is a dict of flag_name to True/False