Skip to content
Software Engineering Hub

Async Programming

Asynchronous programming is a concurrency model built around the idea that tasks should not block while waiting for I/O operations to complete. Instead of tying up a thread while a network request or file read is in progress, async programs register a callback or suspend execution, freeing the runtime to do other useful work in the meantime.

Blocking vs Non-Blocking I/O

The fundamental difference lies in what happens when your program makes an I/O request (reading a file, querying a database, fetching a URL).

Blocking I/O

The calling thread halts until the operation finishes. During that wait, the thread cannot do anything else.

Thread 1:  [── Work ──][──── Waiting for DB ────][── Process Result ──]
Thread 2:  [── Work ──][──── Waiting for API ───][── Process Result ──]
Thread 3:  [── Work ──][──── Waiting for File ──][── Process Result ──]


Each thread is idle while waiting. To handle 1000 concurrent requests,
you need approximately 1000 threads -- expensive and unscalable.

Non-Blocking I/O

The calling thread initiates the operation and moves on to other work. When the result is ready, the program is notified.

Thread 1:  [─ Work ─][Start DB][─ Other Work ─][DB Ready → Process]
           [Start API][─ Other Work ─][API Ready → Process]
           [Start File][─ Other Work ─][File Ready → Process]


One thread handles many concurrent I/O operations by interleaving work.
A single thread can handle thousands of connections.

The Event Loop

The event loop is the heart of async programming. It is a single-threaded loop that continuously checks for completed I/O operations and runs their associated callbacks.

┌─────────────────────────────────────────────┐
│                 EVENT LOOP                   │
│                                             │
│  ┌─────────┐    ┌──────────────────────┐    │
│  │  Timer  │    │   Pending Callbacks   │    │
│  │  Phase  │───►│   (I/O callbacks,     │    │
│  └─────────┘    │    setTimeout, etc.)  │    │
│       ▲         └──────────┬───────────┘    │
│       │                    │                │
│       │                    ▼                │
│  ┌────┴────┐    ┌──────────────────────┐    │
│  │  Close  │    │    Poll Phase         │    │
│  │Callbacks│◄───│  (Retrieve new I/O    │    │
│  └─────────┘    │   events, execute     │    │
│       ▲         │   I/O callbacks)      │    │
│       │         └──────────┬───────────┘    │
│       │                    │                │
│       │                    ▼                │
│  ┌────┴────┐    ┌──────────────────────┐    │
│  │  Check  │◄───│  setImmediate        │    │
│  │  Phase  │    │  callbacks           │    │
│  └─────────┘    └──────────────────────┘    │
│                                             │
│  The loop repeats until there are no more   │
│  callbacks or events to process.            │
└─────────────────────────────────────────────┘

How It Works

  1. The event loop starts and processes any immediately runnable tasks
  2. It checks for completed I/O operations and schedules their callbacks
  3. It runs the queued callbacks one at a time (single-threaded)
  4. It checks for timers that have expired and runs their callbacks
  5. If there is nothing left to do, it waits for new I/O events
  6. Steps 2—5 repeat until the program exits

Because the event loop is single-threaded, a long-running computation in a callback will block the entire loop. This is why CPU-intensive work should be offloaded to worker threads or processes, not run inside the event loop.

The Evolution of Async Patterns

Async programming has evolved through several patterns, each improving on the last.

1. Callbacks

The earliest async pattern: pass a function to be called when the operation completes.

const fs = require('fs');


// Simple callback
fs.readFile('data.txt', 'utf8', (err, data) => {
  if (err) {
    console.error('Error reading file:', err);
    return;
  }
  console.log('File contents:', data);
});


console.log('This runs immediately -- readFile is non-blocking');

Callback Hell

Callbacks become unmanageable when operations depend on each other. Each async step nests inside the previous callback, creating a “pyramid of doom.”

// Callback hell -- deeply nested, hard to read and maintain
getUser(userId, (err, user) => {
  if (err) return handleError(err);
  getOrders(user.id, (err, orders) => {
    if (err) return handleError(err);
    getOrderDetails(orders[0].id, (err, details) => {
      if (err) return handleError(err);
      getShippingStatus(details.trackingId, (err, status) => {
        if (err) return handleError(err);
        console.log('Shipping status:', status);
        // Imagine this going even deeper...
      });
    });
  });
});

This pattern is brittle, hard to debug, and nearly impossible to handle errors correctly across all branches. Promises were invented to solve this problem.

2. Promises and Futures

A Promise (JavaScript) or Future (Python, Java, C++) is an object that represents a value that will be available in the future. Instead of nesting callbacks, you chain operations.

// Promises flatten the callback pyramid
function getUser(userId) {
  return new Promise((resolve, reject) => {
    setTimeout(() => resolve({ id: userId, name: 'Alice' }), 100);
  });
}


function getOrders(userId) {
  return new Promise((resolve, reject) => {
    setTimeout(() => resolve([{ id: 1, item: 'Laptop' }]), 100);
  });
}


function getOrderDetails(orderId) {
  return new Promise((resolve, reject) => {
    setTimeout(() => resolve({ orderId, total: 999, trackingId: 'TRK123' }), 100);
  });
}


// Chained -- flat and readable
getUser(42)
  .then(user => getOrders(user.id))
  .then(orders => getOrderDetails(orders[0].id))
  .then(details => console.log('Order details:', details))
  .catch(err => console.error('Error:', err));  // Single error handler


// Promise.all -- run multiple promises concurrently
Promise.all([
  fetch('https://api.example.com/users'),
  fetch('https://api.example.com/products'),
  fetch('https://api.example.com/orders'),
])
  .then(([users, products, orders]) => {
    console.log('All data fetched');
  })
  .catch(err => console.error('One request failed:', err));


// Promise.race -- resolve as soon as the first one completes
Promise.race([
  fetch('https://api1.example.com/data'),
  fetch('https://api2.example.com/data'),
])
  .then(fastest => console.log('Fastest response:', fastest));

3. Async/Await

Async/await is syntactic sugar over Promises/Futures that makes asynchronous code look and behave like synchronous code. This is the modern standard across most languages.

import asyncio
import aiohttp
import time


# Define a coroutine with 'async def'
async def fetch_url(session, url):
    async with session.get(url) as response:
        return await response.text()


async def main():
    urls = [
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/1",
    ]


    start = time.time()


    async with aiohttp.ClientSession() as session:
        # Sequential -- each request waits for the previous
        for url in urls:
            result = await fetch_url(session, url)
            print(f"Fetched {url[:40]}...")


        print(f"Sequential: {time.time() - start:.2f}s")


    start = time.time()


    async with aiohttp.ClientSession() as session:
        # Concurrent -- all requests run at the same time
        tasks = [fetch_url(session, url) for url in urls]
        results = await asyncio.gather(*tasks)
        for url, result in zip(urls, results):
            print(f"Fetched {url[:40]}...")


        print(f"Concurrent: {time.time() - start:.2f}s")


# asyncio.gather with error handling
async def safe_fetch(session, url):
    try:
        return await fetch_url(session, url)
    except Exception as e:
        return f"Error fetching {url}: {e}"


# Run with timeout
async def fetch_with_timeout():
    try:
        result = await asyncio.wait_for(
            asyncio.sleep(10),  # This will timeout
            timeout=2.0
        )
    except asyncio.TimeoutError:
        print("Operation timed out")


asyncio.run(main())

Coroutines

Coroutines are functions that can be suspended and resumed. Unlike regular functions that run from start to finish, a coroutine can pause at specific points (e.g., await, yield) and give control back to the caller or event loop. When the awaited operation completes, the coroutine resumes from where it left off.

Regular Function:           Coroutine:


  start ──► run ──► end      start ──► run ──► suspend
                                                  │
                              (event loop does     │
                               other work)         │
                                                  │
                              resume ◄─────────────┘
                                │
                              run ──► suspend
                                        │
                              resume ◄──┘
                                │
                              run ──► end

Cooperative vs Preemptive Scheduling

AspectCooperative (Coroutines)Preemptive (OS Threads)
Who decides when to switch?The task yields control explicitly (await, yield)The OS scheduler forces context switches at any time
OverheadVery low (no kernel involvement)Higher (kernel context switch, register save/restore)
DeterminismHigh — you control when switches happenLow — the OS can switch at any instruction
RiskA misbehaving coroutine can block the loopThe OS ensures fairness even with misbehaving threads
Concurrency bugsFewer race conditions (single-threaded)Many possible race conditions
Best forI/O-bound workloadsCPU-bound and mixed workloads

When to Use Async vs Threads

Choosing between async programming and threading depends on the nature of your workload.

Use Async When

  • Your application is I/O-bound (network calls, database queries, file reads)
  • You need to handle many concurrent connections (web servers, API gateways)
  • You want to minimize memory usage (coroutines use far less memory than threads)
  • You are working in an ecosystem with good async support (Node.js, Python asyncio, Rust Tokio)

Use Threads When

  • Your application is CPU-bound (image processing, scientific computing, encryption)
  • You need true parallelism on multiple cores
  • You are using libraries that block and do not have async alternatives
  • Your language’s async runtime has limitations (Python’s GIL limits thread-based parallelism for CPU work, but multiprocessing bypasses it)

The Hybrid Approach

In practice, many applications combine both:

┌───────────────────────────────────────────────────────┐
│                  Application                           │
│                                                       │
│  ┌─────────────────────────────────────────────────┐  │
│  │           Async Event Loop (Main)                │  │
│  │                                                 │  │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐        │  │
│  │  │ HTTP Req │ │ DB Query │ │ WebSocket│        │  │
│  │  │ (async)  │ │ (async)  │ │ (async)  │        │  │
│  │  └──────────┘ └──────────┘ └──────────┘        │  │
│  │                                                 │  │
│  │  CPU-heavy tasks offloaded to thread pool:      │  │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐        │  │
│  │  │  Image   │ │   Data   │ │  Crypto  │        │  │
│  │  │ Process  │ │  Crunch  │ │  Hash    │        │  │
│  │  │ (thread) │ │ (thread) │ │ (thread) │        │  │
│  │  └──────────┘ └──────────┘ └──────────┘        │  │
│  └─────────────────────────────────────────────────┘  │
└───────────────────────────────────────────────────────┘
import asyncio
import concurrent.futures
import hashlib


# CPU-bound work -- offload to thread pool
def cpu_intensive_hash(data):
    """Simulate CPU-bound work"""
    for _ in range(1000000):
        data = hashlib.sha256(data).digest()
    return data.hex()


async def main():
    loop = asyncio.get_event_loop()


    # Create a thread pool for CPU-bound work
    with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool:
        # Run CPU-bound work in the thread pool,
        # without blocking the event loop
        result = await loop.run_in_executor(
            pool,
            cpu_intensive_hash,
            b"hello world"
        )
        print(f"Hash result: {result[:20]}...")


        # Meanwhile, async I/O operations still run concurrently
        # on the event loop
        await asyncio.sleep(0.1)  # Simulate async I/O


asyncio.run(main())

Common Async Patterns

Pattern 1: Fan-Out / Fan-In

Launch many concurrent operations and collect all results.

import asyncio


async def fetch_page(page_num):
    await asyncio.sleep(0.5)  # Simulate network request
    return {"page": page_num, "items": [f"item-{page_num}-{i}" for i in range(10)]}


async def fetch_all_pages(total_pages):
    # Fan-out: launch all requests concurrently
    tasks = [fetch_page(i) for i in range(1, total_pages + 1)]


    # Fan-in: collect all results
    results = await asyncio.gather(*tasks)
    all_items = [item for result in results for item in result["items"]]
    return all_items


asyncio.run(fetch_all_pages(10))

Pattern 2: Rate-Limited Concurrency

Limit how many async operations run concurrently to avoid overwhelming a server or exceeding API rate limits.

import asyncio


async def fetch_with_limit(url, semaphore):
    async with semaphore:
        print(f"Fetching: {url}")
        await asyncio.sleep(1)  # Simulate request
        return f"Result from {url}"


async def main():
    # Allow at most 5 concurrent requests
    semaphore = asyncio.Semaphore(5)


    urls = [f"https://api.example.com/item/{i}" for i in range(20)]
    tasks = [fetch_with_limit(url, semaphore) for url in urls]
    results = await asyncio.gather(*tasks)
    print(f"Fetched {len(results)} items")


asyncio.run(main())

Pattern 3: Timeout and Cancellation

Set deadlines on async operations so they do not hang indefinitely.

import asyncio


async def slow_operation():
    await asyncio.sleep(10)
    return "completed"


async def main():
    # Timeout after 2 seconds
    try:
        result = await asyncio.wait_for(slow_operation(), timeout=2.0)
        print(result)
    except asyncio.TimeoutError:
        print("Operation timed out!")


    # Cancellation
    task = asyncio.create_task(slow_operation())
    await asyncio.sleep(1)
    task.cancel()


    try:
        await task
    except asyncio.CancelledError:
        print("Task was cancelled")


asyncio.run(main())

Comparison Table: Async Across Languages

FeaturePythonJavaScriptJavaC++
Async keywordasync defasync functionCompletableFuture / virtual threadsco_await (C++20)
Await keywordawaitawait.thenCompose() / .get()co_await
Event loopasyncioBuilt-in (libuv in Node.js)Netty, Vert.x (external)Boost.Asio (external)
Concurrency primitiveasyncio.gather()Promise.all()CompletableFuture.allOf()std::async
Cancellationtask.cancel()AbortControllerfuture.cancel()Cooperative (no built-in)
Timeoutasyncio.wait_for()AbortController + setTimeout.orTimeout()Custom with std::future
Ecosystemaiohttp, FastAPIExpress, FastifySpring WebFlux, QuarkusBoost.Asio, cppcoro

Common Mistakes

  1. Forgetting to await — The function returns a coroutine/promise object instead of the result. In Python, this produces a warning; in JavaScript, you silently get a Promise instead of the value.

  2. Blocking the event loop — Calling time.sleep() instead of await asyncio.sleep(), or running CPU-heavy code directly in an async function. This freezes all concurrent operations.

  3. Creating tasks without awaiting them — “Fire and forget” tasks whose exceptions go unhandled. Always gather or await your tasks.

  4. Sequential awaits when concurrent is possible — Writing a = await x; b = await y when x and y are independent. Use gather or Promise.all instead.

  5. Mixing sync and async incorrectly — Calling async functions from synchronous code (or vice versa) without proper bridging.

Next Steps

Race Conditions & Common Pitfalls Learn to identify and prevent race conditions, deadlocks, and other concurrency bugs
Threads & Synchronization Revisit thread-based concurrency and synchronization primitives
Concurrency & Parallelism Overview Return to the section overview for key concepts and concurrency models