datacube.utils.dask.partition_map#

datacube.utils.dask.partition_map(n, func, its, name='compute')[source]#

Parallel map in lumps.

Partition sequence into lumps of size n, then construct dask delayed computation evaluating to:

[func(x) for x in its[0:1n]],
[func(x) for x in its[n:2n]],
...
[func(x) for x in its[..]],

This is useful when you need to process a large number of small (quick) tasks (pixel drill for example).

Parameters
  • n (int) – number of elements to process in one go

  • func (Any) – Function to apply (non-dask)

  • its (Iterable[Any]) – Values to feed to fun

  • name (str) – How the computation should be named in dask visualizations

Return type

Iterator of dask.Delayed objects.

Return type

Iterable[Any]