Exponential Backoff: How to Actually Retry Something

A follow-up to my last post on circuit breakers. That one was about when to stop trying. This one's about how to try again, properly.

Quick recap from last time

Circuit breakers stop you from calling something that's already failing over and over. Good. But eventually, you do want to try again. The question this post answers: when you retry, how do you actually space it out?

The naive way, and why it's bad

Say a call fails. The easy fix is: just try again, right away.

Sounds fine for one request. Now imagine a hundred requests all failed at the same time, because the other service had a small hiccup. All hundred retry immediately, at the same moment, hitting the already-struggling service all over again, at once. You just made the hiccup worse.

The idea: wait longer each time

Instead of retrying immediately, you wait a bit. If it fails again, you wait longer. And longer again. That's it. That's exponential backoff.

• 1st retry: wait 1 second

• 2nd retry: wait 2 seconds

• 3rd retry: wait 4 seconds

• 4th retry: wait 8 seconds

Each wait roughly doubles. Give the other service breathing room instead of hammering it again right away.

One more small trick: jitter

If a hundred requests all failed at once, and they all use the exact same doubling pattern, they're still all retrying at the same moments together. So you add a little randomness to the wait time, this is called jitter. Instead of everyone waiting exactly 2 seconds, everyone waits somewhere around 2 seconds, spread out a bit. Now they don't all slam the service at the same instant again.

A small example in code

import time
import random

def call_with_backoff(func, max_retries=4):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception:
            if attempt == max_retries - 1:
                raise  # out of retries, give up for real

            wait = (2 ** attempt) + random.uniform(0, 1)  # doubling + jitter
            print(f"Failed, waiting {wait:.1f}s before retry")
            time.sleep(wait)

Using it:

call_with_backoff(lambda:requests.get("https://example.com/api/data"))

First failure waits about 1 second. Second failure waits about 2. Third about 4. Each with a bit of random extra sprinkled in, so it's not perfectly predictable.

Putting them together in one bit of code

Here's both ideas from the last two posts, combined. The circuit breaker decides if it's even worth trying. Backoff decides how to space out the attempts if it is:

import time
import random

class CircuitBreaker:
    def __init__(self, max_failures=3, wait_time=10):
        self.failures = 0
        self.max_failures = max_failures
        self.wait_time = wait_time
        self.state = "closed"
        self.opened_at = None

    def allow_request(self):
        if self.state == "open":
            if time.time() - self.opened_at > self.wait_time:
                self.state = "half-open"
                return True
            return False
        return True

    def record_success(self):
        self.failures = 0
        self.state = "closed"

    def record_failure(self):
        self.failures += 1
        if self.failures >= self.max_failures:
            self.state = "open"
            self.opened_at = time.time()


def call_with_backoff_and_breaker(func, breaker, max_retries=4):
    for attempt in range(max_retries):
        if not breaker.allow_request():
            raise Exception("Circuit is open — not even trying")

        try:
            result = func()
            breaker.record_success()
            return result
        except Exception:
            breaker.record_failure()
            if attempt == max_retries - 1:
                raise

            wait = (2 ** attempt) + random.uniform(0, 1)
            time.sleep(wait)

Using it:

breaker = CircuitBreaker()
call_with_backoff_and_breaker(
    lambda: requests.get("https://example.com/api/data"),
    breaker,
)

Every attempt first checks with the breaker: "should I even try?" If yes, it tries, and waits a bit longer each time it fails, same doubling-plus-jitter as before. If the breaker trips partway through, the retries stop immediately instead of continuing to wait and fail.

How this connects to the circuit breaker

They work together, not instead of each other. Backoff controls how you retry. The circuit breaker controls whether you should even be trying at all right now. A real setup usually has both: back off between retries, and if it keeps failing past a certain point, let the circuit breaker trip and stop trying entirely for a while.

The one-line version

Don't retry instantly. Wait a bit, then a bit more each time, and add a small random amount so everyone retrying isn't doing it at the exact same second. Find me on LinkedIn 😄.