What is asyncio.gather()?
The asyncio library in Python is very popular for writing asynchronous code using async/wait syntax. It is mostly present and comes with the recent Python versions (3.5 and higher).
In this Answer, we will learn about the asyncio.gather() function.
Introduction to asyncio.gather() function
The asyncio.gather() is a function provided by the Python asyncio library that allows multiple coroutines to be executed concurrently and wait for all of them to complete before continuing. It is a powerful tool for concurrent programming in Python that enables us to execute multiple tasks in parallel and retrieve their results easily.
When we call asyncio.gather(), it takes a list of coroutines as input and returns a coroutine that we can await. When we await this coroutine, it will execute all the coroutines in the list concurrently and wait for them to complete. The return value of asyncio gathers a list of results from each coroutine in the input list in the order in which they were specified.
How asyncio.gather() works
The asyncio.gather() function in Python works by executing multiple coroutines concurrently and waiting for them to complete before returning their results. Here's how it works in more detail:
When we call the
asyncio.gather()function, it takes a list of coroutines as input. These coroutines represent the tasks that we want to execute concurrently.This function creates a task for each coroutine in the input list. These tasks are added to the event loop, which is responsible for executing the tasks.
As the event loop runs, it schedules each task to run concurrently with the other tasks in the list. This means that the coroutines are executing in parallel rather than sequentially.
Once all the coroutines have been completed, the
asyncio.gather()function returns a list of their results. The results are returned in the same order as the coroutines in the input list.If any coroutines raise an exception, then the function will immediately cancel all remaining tasks and raise the exception. This ensures that errors are handled quickly and efficiently.
Syntax for asyncio.gather()
results = await asyncio.gather(*coroutines,return_exceptions = False,timeout = None,cancel_on_error = False)
The first argument is a sequence of coroutines to be executed concurrently.
The
*operator unpacks the sequence of coroutines into individual arguments, allowing them to be passed to the gather function as separate arguments.The
return_exceptionsparameter is an optional boolean parameter that specifies whether exceptions raised by coroutines should be returned in the results list or raised as exceptions.The
timeoutparameter is an optional float that specifies the maximum time (in seconds) the entire operation should take before raising aTimeoutErrorexception.The
cancel_on_errorparameter is an optional boolean parameter that specifies whether to cancel remaining coroutines if an error occurs during execution.
Coding example
The coding example for asyncio.gather() is as follows:
# import the asyncio libraryimport asyncio# define a TimeCount function using asyncasync def TimeCount():print("It is 1'oclock")await asyncio.sleep(1)print("It is 2'oclock")await asyncio.sleep(1)print("It is 3'oclock")await asyncio.sleep(1)print("It is 4'oclock")await asyncio.sleep(1)print("It is 5'oclock")# lets create a main functionasync def main():# we will pass the TimeCount() into the asyncio gather functioawait asyncio.gather(TimeCount())if __name__ == "__main__":import timetpc = time.perf_counter()asyncio.run(main())Timeup = time.perf_counter() - tpcprint(f"{__file__} executed in {Timeup:0.2f} seconds.")
Code explanation
Here is the explanation of the code:
Line 2: We import the
asynciolibrary.Line 5: We create a
TimeCount()function using async as pass in some print statements.Line 16: The code defines an async function called
main(), which is decorated with the async keyword.Line 18: The await keyword is used to wait for the completion of
asyncio.gather(TimeCount())expression, which waits for theTimeCount()coroutine to complete execution.Line 20: The
if __name__ == "__main__":statement is a Python idiom that checks whether the script is being run as the main program. This allows the code inside theifblock to be executed only when the script is run directly, as opposed to when it is imported as a module into another script.Line 21: The
Import timestatement imports the time module.Line 22: We create a variable
tpcand assign to to thetime.perf_counter()statement, which records the start time of the script using theperf_counter()method of the time module returns the current time in seconds with high resolution.Line 23: The
asyncio.run(main())statement invokes themain()function using therun()function of the asyncio module, which starts the event loop and runs themain()function until it completes.Line 24: We calculate the elapsed time of the script by subtracting the start time recorded earlier from the current time using the
perf_counter()method of the time module.Line 25: This statement prints the name of the current script and the elapsed time in seconds with two decimal places using string interpolation.
Best practices for using asyncio.gather()
Following are a few best practices to use asyncio.gather() function:
Design coroutines for concurrency: The key to making the most of
asyncio.gather()is to write coroutines that can execute concurrently. We must avoid blocking I/O calls and use asyncio functions to perform I/O operations asynchronously.Tune the parameters for optimal performance: The
asyncio.gather()has several parameters that can be used to customize its behavior, including thereturn_exceptions,timeout,cancel_on_error, and result filter functions. Careful tuning of these parameters can help us achieve the best possible performance for our specific use case.Proper exception handling and cancellation: When using
asyncio.gather(), handling exceptions properly and canceling tasks when necessary is important. We usetry/exceptblocks to catch and handle exceptions our coroutines may raise. We must also cancel all tasks using theasyncio.CancelledErrorexception when the gather operation is canceled.Use
asyncio.gather()only when necessary: Whileasyncio.gather()is a powerful tool for executing multiple coroutines concurrently, it's not always the best solution for every use case. Before usingasyncio.gather(), we should consider whether there are simpler and more efficient ways to achieve our goals, such as using async for loops or other asyncio constructs.Use
asyncio.gather()with other asyncio constructs: Theasyncio.gather()is just one of many tools in the asyncio toolbox. To make the most of asyncio, we should consider using asyncio gather in conjunction with other asyncio constructs like asyncio locks, queues, and semaphores.
Advanced features of asyncio.gather()
The following are the advanced features of the asyncio.gather() function:
Return_exceptionsparameter: By default, if any of the coroutines passed toasyncio.gather()raises an exception, the entire gather operation is aborted, and the exception is raised.Timeoutparameter: We can use the timeout parameter to specify a timeout value for the entireasyncio.gather()operation. If any coroutines take longer than the specified timeout,asyncio.gather()will raise aTimeoutErrorexception and cancel all the remaining ones.Cancel_on_errorparameter: Thecancel_on_errorparameter can specify whetherasyncio.gather()should cancel all the remaining coroutines if any of them raises an exception. By default, this parameter is set toTrue, which means thatasyncio.gather()will cancel all the remaining coroutines if any of them raise an exception.Result filter functions: We can pass a filter function to
asyncio.gather()as a keyword argument to filter the results of the coroutines. It can exclude certain values or handle the results in a specific way.
Conclusion
In this Answer, we discussed the asyncio.gather() function; how it works, its syntax, some best practices of using this function, advanced features, and finally, we considered an example code.
