Python asyncio

last modified October 30, 2026 Python 3.15

In this article we show how to use the asyncio module for asynchronous programming in Python.

With asynchronous programming, we can execute many tasks concurrently in a single thread. The syntax of coroutines, async and await, is covered in Python async/await; here we focus on the tools of the asyncio module.

The asyncio module

The asyncio module is a library to write asynchronous programs in Python. It provides high-level APIs to:

• run Python coroutines concurrently and have full control over their execution
• perform network IO and IPC
• control subprocesses
• distribute tasks via queues
• synchronize concurrent code

Concurrent programming is used for two kinds of tasks: IO-bound tasks and CPU-bound tasks. Requesting data from a network, accessing a database, or reading and writing files are IO-bound tasks. CPU-bound tasks are computationally expensive, such as mathematical calculations or graphics processing.

Asynchronous programs consist of an event loop, coroutines, tasks, and futures. The event loop is the scheduler: it runs tasks, waits for IO, and resumes the tasks whose data is ready. Coroutines are functions that can be paused at await expressions. A task wraps a coroutine and schedules it on the event loop. A future represents the eventual result of an operation; the result may also be an exception.

Python asyncio simple example

The next is a simple example with asyncio.

import asyncio


async def mul(x, y):
    return x * y


res = asyncio.run(mul(5, 5))
print(res)

The program runs a coroutine which multiplies two numbers.

res = asyncio.run(mul(5, 5))

The asyncio.run function creates a new event loop, runs the coroutine until it completes, closes the loop, and returns the result. It should be the main entry point of asyncio programs and should be called only once.

$ python simple.py
25

Event loops and runners

Older code creates the event loop with asyncio.get_event_loop. Since Python 3.14, the function raises a RuntimeError when there is no running or explicitly set event loop.

import asyncio


async def mul(x, y):
    return x * y


try:
    loop = asyncio.get_event_loop()
except RuntimeError as e:
    print(f'RuntimeError: {e}')

loop = asyncio.new_event_loop()

try:
    print(loop.run_until_complete(mul(5, 5)))
finally:
    loop.close()

with asyncio.Runner() as runner:
    print(runner.run(mul(6, 6)))
    print(runner.run(mul(7, 7)))

The example shows the low-level API and the asyncio.Runner class.

loop = asyncio.new_event_loop()

try:
    print(loop.run_until_complete(mul(5, 5)))
finally:
    loop.close()

The new_event_loop function creates a loop explicitly. The run_until_complete method runs the loop until the coroutine is done and returns its result. The loop must be closed.

with asyncio.Runner() as runner:
    print(runner.run(mul(6, 6)))
    print(runner.run(mul(7, 7)))

The asyncio.Runner context manager (Python 3.11) runs several coroutines in the same event loop and closes it at the end. It replaces the low-level code in most cases.

$ python event_loop.py
RuntimeError: There is no current event loop in thread 'MainThread'.
25
36
49

Python asyncio tasks

The create_task function wraps the given coroutine into a task and schedules its execution on the running event loop. It returns the task object. Tasks give us additional functionality, such as cancellation or checking the ready status.

import asyncio
import time


async def task(tid, n):
    await asyncio.sleep(n)
    print(f'task {tid} finished')
    return tid


async def main():
    t1 = asyncio.create_task(task(1, 3))
    t2 = asyncio.create_task(task(2, 2))
    t3 = asyncio.create_task(task(3, 1))

    print(t1.get_name(), t1.done())

    res = await asyncio.gather(t1, t2, t3)
    print(res, t1.done())


tm1 = time.perf_counter()

asyncio.run(main())

tm2 = time.perf_counter()
print(f'Total time elapsed: {tm2 - tm1:0.2f} seconds')

In the example, we create three tasks that simulate some work with asyncio.sleep. The asyncio.gather function waits for all of them and returns their results in order.

print(t1.get_name(), t1.done())

Each task has a name, which is useful for debugging. A task created this way has not started yet; it runs when the main coroutine awaits.

The tasks run concurrently, so the total time equals the longest task. We measure the elapsed time with time.perf_counter.

$ python tasks.py
Task-2 False
task 3 finished
task 2 finished
task 1 finished
[1, 2, 3] True
Total time elapsed: 3.00 seconds

Python asyncio as_completed

The asyncio.as_completed function yields the awaitables in the order they complete, so we can process each result as soon as it is available. Since Python 3.13, it can be used with async for.

import asyncio


async def download(name, n):
    await asyncio.sleep(n)
    return f'{name} ({n} s)'


async def main():
    jobs = [download('video', 0.3), download('image', 0.1),
            download('text', 0.2)]

    async for task in asyncio.as_completed(jobs):
        print(await task)


asyncio.run(main())

We start three downloads and print each one when it is finished. The results come in the order of their durations, not in the order of the list.

$ python as_completed.py
image (0.1 s)
text (0.2 s)
video (0.3 s)

Python asyncio wait

The asyncio.wait function waits for a set of tasks and returns two sets: the done tasks and the pending tasks. The return_when parameter determines when it returns: FIRST_COMPLETED, FIRST_EXCEPTION, or ALL_COMPLETED.

import asyncio


async def resolve(server, n):
    await asyncio.sleep(n)
    return server


async def main():
    tasks = [asyncio.create_task(resolve(server, n))
             for server, n in [('dns1', 0.3), ('dns2', 0.1), ('dns3', 0.2)]]

    done, pending = await asyncio.wait(tasks,
                                       return_when=asyncio.FIRST_COMPLETED)

    print(f'done: {[task.result() for task in done]}')
    print(f'pending: {len(pending)}')

    for task in pending:
        task.cancel()

    await asyncio.wait(pending)
    print([task.cancelled() for task in tasks])


asyncio.run(main())

We ask three servers and use the first answer.

for task in pending:
    task.cancel()

await asyncio.wait(pending)

The wait function does not cancel the pending tasks; we must do it ourselves. We then wait for the cancellation to complete. Only the fastest task finished normally.

$ python wait_first.py
done: ['dns2']
pending: 2
[True, False, True]

Python asyncio queue

The asyncio.Queue distributes work between producers and consumers. A queue with maxsize makes the producer wait when the queue is full. Since Python 3.13, the shutdown method tells the consumers that no more items will come.

import asyncio
import random


async def producer(queue, n):
    for i in range(n):
        await queue.put(f'job {i}')
    queue.shutdown()


async def worker(name, queue, results):
    while True:
        try:
            job = await queue.get()
        except asyncio.QueueShutDown:
            break

        await asyncio.sleep(random.uniform(0.1, 0.3))
        results.append((name, job))
        queue.task_done()


async def main():
    queue = asyncio.Queue(maxsize=3)
    results = []

    async with asyncio.TaskGroup() as tg:
        tg.create_task(producer(queue, 8))
        for name in ['w1', 'w2', 'w3']:
            tg.create_task(worker(name, queue, results))

    print(len(results))

    for name in ['w1', 'w2', 'w3']:
        jobs = [job for w, job in results if w == name]
        print(name, len(jobs))


asyncio.run(main())

One producer puts eight jobs into the queue and three workers process them. Each job takes a random amount of time.

try:
    job = await queue.get()
except asyncio.QueueShutDown:
    break

After shutdown, the get method still returns the remaining items; when the queue is empty, it raises QueueShutDown, and the worker ends. Without shutdown, the workers would wait forever and we would have to cancel them.

async with asyncio.TaskGroup() as tg:

The task group waits until the producer and all workers finish. The jobs are divided between the workers depending on how fast they are.

$ python queue_workers.py
8
w1 3
w2 3
w3 2

Limiting concurrency with a semaphore

The asyncio.Semaphore limits the number of coroutines that can run a block of code at the same time. It is useful, for example, to limit the number of simultaneous requests to a server.

import asyncio
import time


async def fetch(n, sem):
    async with sem:
        print(f'{time.perf_counter() - start:0.1f} s: fetching {n}')
        await asyncio.sleep(1)


async def main():
    sem = asyncio.Semaphore(2)
    await asyncio.gather(*(fetch(n, sem) for n in range(6)))


start = time.perf_counter()
asyncio.run(main())
print(f'total: {time.perf_counter() - start:0.1f} s')

We start six fetches, but the semaphore allows only two at a time.

async with sem:

The async with statement acquires the semaphore and releases it at the end of the block. The fetches run in pairs, so the total time is three seconds.

$ python semaphore.py
0.0 s: fetching 0
0.0 s: fetching 1
1.0 s: fetching 2
1.0 s: fetching 3
2.0 s: fetching 4
2.0 s: fetching 5
total: 3.0 s

Race conditions and locks

Asyncio code runs in a single thread, but race conditions can still occur. When a coroutine reads shared state, awaits, and then writes the state, other coroutines can change the state in between.

import asyncio

balance = 0


async def deposit(amount):
    global balance

    current = balance
    await asyncio.sleep(0.01)
    balance = current + amount


async def safe_deposit(amount, lock):
    global balance

    async with lock:
        current = balance
        await asyncio.sleep(0.01)
        balance = current + amount


async def main():
    global balance

    await asyncio.gather(*(deposit(10) for _ in range(100)))
    print(f'without lock: {balance}')

    balance = 0
    lock = asyncio.Lock()
    await asyncio.gather(*(safe_deposit(10, lock) for _ in range(100)))
    print(f'with lock: {balance}')


asyncio.run(main())

We deposit ten units one hundred times, concurrently.

current = balance
await asyncio.sleep(0.01)
balance = current + amount

All coroutines read the balance before any of them writes it, so all but one deposit are lost.

async with lock:

The asyncio.Lock lets only one coroutine at a time into the block, and the result is correct.

$ python lock.py
without lock: 10
with lock: 1000

Python asyncio subprocesses

The create_subprocess_exec and create_subprocess_shell functions start processes without blocking the event loop.

import asyncio
import time


async def run(cmd):
    proc = await asyncio.create_subprocess_shell(
        cmd, stdout=asyncio.subprocess.PIPE)

    stdout, _ = await proc.communicate()
    return proc.returncode, stdout.decode().strip()


async def main():
    cmds = ['sleep 1; echo first', 'sleep 1; echo second',
            'sleep 1; echo third', 'exit 3']

    res = await asyncio.gather(*(run(cmd) for cmd in cmds))

    for cmd, (code, out) in zip(cmds, res):
        print(f'{cmd!r:25} exit={code} output={out!r}')


start = time.perf_counter()
asyncio.run(main())
print(f'elapsed: {time.perf_counter() - start:0.2f} s')

We run four shell commands concurrently.

proc = await asyncio.create_subprocess_shell(
    cmd, stdout=asyncio.subprocess.PIPE)

stdout, _ = await proc.communicate()

The standard output is redirected to a pipe. The communicate method waits for the process to finish and returns its output. Three commands sleep for one second, but the whole program takes about one second.

$ python subprocesses.py
'sleep 1; echo first'     exit=0 output='first'
'sleep 1; echo second'    exit=0 output='second'
'sleep 1; echo third'     exit=0 output='third'
'exit 3'                  exit=3 output=''
elapsed: 1.00 s

Python asyncio echo server

In the following example, we create an echo server with asyncio streams. An echo server sends the message from the client back.

import asyncio


async def handle(reader, writer):
    host, _ = writer.get_extra_info('peername')

    while line := await reader.readline():
        print(f'received {line!r} from {host}')
        writer.write(line)
        await writer.drain()

    writer.close()
    await writer.wait_closed()


async def main(host, port):
    server = await asyncio.start_server(handle, host, port)
    print(f'serving on {host}:{port}')

    async with server:
        await server.serve_forever()


try:
    asyncio.run(main('127.0.0.1', 8001))
except KeyboardInterrupt:
    print('terminated')

The start_server function starts a TCP server and calls the handle coroutine for each client connection, with a reader and a writer stream.

while line := await reader.readline():
    print(f'received {line!r} from {host}')
    writer.write(line)
    await writer.drain()

We read the data line by line and write it back. The drain method waits until the write buffer is flushed. The readline method returns an empty bytes object when the client closes the connection.

async with server:
    await server.serve_forever()

The serve_forever method accepts connections until the coroutine is cancelled. Pressing Ctrl+C raises KeyboardInterrupt from asyncio.run.

Next, we create a client for the server.

import asyncio


async def main():
    reader, writer = await asyncio.open_connection('127.0.0.1', 8001)

    for msg in ['Hello!', 'Hi!']:
        writer.write(f'{msg}\n'.encode())
        await writer.drain()

        data = await reader.readline()
        print(f'echo: {data.decode().strip()}')

    writer.close()
    await writer.wait_closed()


asyncio.run(main())

The open_connection function connects to the server and returns a reader and a writer. We send two lines and read the replies.

We start the server in the background, and connect to it with the client and with the nc tool. The server prints the received lines. Finally, we stop the server with Ctrl+C.

$ python echo_server.py &
serving on 127.0.0.1:8001
$ python echo_client.py
echo: Hello!
echo: Hi!
received b'Hello!\n' from 127.0.0.1
received b'Hi!\n' from 127.0.0.1
$ echo 'Hello from nc' | nc -q 1 127.0.0.1 8001
Hello from nc
received b'Hello from nc\n' from 127.0.0.1
terminated

Cancelling a task group

Since Python 3.15, the TaskGroup.cancel method cancels all tasks of a task group. The async with block then exits without raising an exception. It is useful for tasks that run forever.

import asyncio
import itertools


async def sensor(name, delay, readings):
    for i in itertools.count():
        await asyncio.sleep(delay)
        readings.append(f'{name}{i}')


async def monitor(readings, tg):
    while len(readings) < 5:
        await asyncio.sleep(0.05)
    print('enough readings, stopping')
    tg.cancel()


async def main():
    readings = []

    async with asyncio.TaskGroup() as tg:
        tg.create_task(sensor('a', 0.2, readings))
        tg.create_task(sensor('b', 0.3, readings))
        tg.create_task(monitor(readings, tg))

    print(readings)


asyncio.run(main())

Two sensors produce readings in endless loops. The monitor task checks the number of readings.

print('enough readings, stopping')
tg.cancel()

When there are five readings, the monitor cancels the whole group, including itself. Before Python 3.15, we had to keep references to the tasks and cancel them one by one, or raise a custom exception and catch it outside the group. See Python async/await for a comparison with Python 3.14.

$ python group_cancel.py
enough readings, stopping
['a0', 'b0', 'a1', 'a2', 'b1']

Multiple HTTP requests

To make multiple asynchronous HTTP requests, we use the httpx module, installed with pip install httpx. The module provides an HTTP client with sync and async APIs, and support for both HTTP/1.1 and HTTP/2.

import asyncio
import time

import httpx

urls = ['https://example.com', 'https://httpbin.org/get',
        'https://www.python.org', 'https://github.com', 'https://pypi.org']


async def get_status(client, url):
    start = time.perf_counter()
    resp = await client.get(url)
    return url, resp.status_code, time.perf_counter() - start


async def main():
    async with httpx.AsyncClient(follow_redirects=True) as client:
        res = await asyncio.gather(*(get_status(client, url) for url in urls))

    for url, status, elapsed in res:
        print(f'{url:25} {status} {elapsed:0.2f} s')


start = time.perf_counter()
asyncio.run(main())
print(f'total: {time.perf_counter() - start:0.2f} s')

The example makes asynchronous HTTP requests to five websites. It prints the status code and the time of each request.

async with httpx.AsyncClient(follow_redirects=True) as client:
    res = await asyncio.gather(*(get_status(client, url) for url in urls))

All requests share one client, which reuses connections. The requests run concurrently, so the total time is close to the time of the slowest request. See Python httpx.

$ python async_req.py
https://example.com       200 0.24 s
https://httpbin.org/get   200 0.62 s
https://www.python.org    200 0.30 s
https://github.com        200 0.51 s
https://pypi.org          200 0.24 s
total: 0.65 s

Source

Python asyncio - Python documentation

In this article we have worked with the Python asyncio module.

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all Python tutorials.