datacube.utils.dask.compute_tasks#
- datacube.utils.dask.compute_tasks(tasks, client, max_in_flight=3)[source]#
Parallel compute stream with back pressure.
Equivalent to:
(client.compute(task).result() for task in tasks)
but with up to
max_in_flight
tasks being processed at the same time. Input/Output order is preserved, so there is a possibility of head of line blocking. :rtype:Iterable
[Any
]Note
lower limit is 3 concurrent tasks to simplify implementation, there is no point calling this function if you want one active task and supporting exactly 2 active tasks is not worth the complexity, for now. We might special-case 2 at some point.