Exercise: Async Python
Exercise 1: Run API Calls Concurrently
Take three simulated API calls that currently run one after another and make them run concurrently with asyncio.gather, so the total time is set by the slowest call rather than the sum of all three.
When an application makes several independent model calls, running them sequentially wastes time: each call spends most of its life waiting on the network while the CPU sits idle. asyncio.gather schedules the calls together and lets the event loop overlap that waiting, which is one of the simplest, highest-impact speedups in an AI application.
Requirements
- Keep the
call_llmcoroutine, which simulates a network call withasyncio.sleep. - Run all three calls concurrently with
asyncio.gatherinstead of awaiting them one at a time. - Print each result and the total elapsed time.
- The elapsed time should be close to the longest single delay (about 0.3s), not their sum (0.6s).
Expected Output
All three responses print, and the total time is close to the single longest delay rather than the sum.
Solution
asyncio.gather schedules all three coroutines on the event loop at once and waits for them together. While each one is parked on await asyncio.sleep, the loop runs the others, so the three waits overlap and the total time collapses to the longest single delay. The returned list preserves the order of the arguments, which is why you can match results back to their prompts without extra bookkeeping.
Exercise 2: Handle Failures in Concurrent Calls
When you fan out many calls at once, some will fail while others succeed. By default asyncio.gather cancels everything if one task raises. Use return_exceptions=True so the gather completes, collecting each task's result or its exception, and then handle them individually.
This is the difference between a fragile batch (one failure loses every result) and a resilient one (you keep the successes and deal with the failures). It is the standard pattern for running independent model or API calls in parallel.
Requirements
- Keep the
call_llmcoroutine, where one call is set to fail. - Run all three concurrently with
asyncio.gather(..., return_exceptions=True). - Iterate the results, printing successes and failures separately by checking
isinstance(result, Exception).
Expected Output
The two good calls return their results; the failing one is reported, not raised, and does not lose the others.
Solution
return_exceptions=True changes gather from all-or-nothing to collect-everything: a raised exception becomes an item in the results list instead of propagating and cancelling the siblings. Checking isinstance(result, Exception) lets you separate the successes from the failures and decide what to do with each, such as retrying only the failed calls. The results stay in the order you submitted the tasks, so you can map them back to their inputs.