compute_tasks(tasks, client, max_in_flight=3)¶
Parallel compute stream with back pressure.
(client.compute(task).result() for task in tasks)
but with up to
max_in_flighttasks being processed at the same time. Input/Output order is preserved, so there is a possibility of head of line blocking.
lower limit is 3 concurrent tasks to simplify implementation, there is no point calling this function if you want one active task and supporting exactly 2 active tasks is not worth the complexity, for now. We might special-case 2 at some point.