Timeouts and retries

Networks and services fail for a moment all the time. A good client is ready for it.

Timeouts

A timeout is the longest you will wait for an answer, like 2 seconds. Without one, a call to a frozen service can hang forever. Each stuck call holds a thread or a connection, and soon the caller runs out and fails too.

A timeout does not mean the work failed. The server may have finished it, and only the answer got lost.

When to retry

Many failures are brief: a dropped connection, a timeout, a 503 Service Unavailable. Trying again often works. Some errors repeat every time, like 400 Bad Request or 404 Not Found. Don't retry those.

Keep the number of tries small, like 3. Endless retries pile load on a service that is already struggling.

Backoff and jitter

Exponential backoff doubles the wait each time: 1 s, 2 s, 4 s, 8 s. It gives the service time to recover.

If a thousand clients fail at the same moment, they also retry at the same moments, in waves. Jitter adds a random amount to each wait, which spreads them out.

Idempotency

An action is idempotent if doing it twice has the same effect as doing it once. "Set the price to 10" is idempotent. "Add 10 to the balance" is not.

In HTTP, GET, PUT and DELETE are idempotent, so they are safe to retry. POST usually creates something or does an action, so a retry can do it twice.

Idempotency keys

To retry a POST safely, like a payment, the client makes a unique idempotency key for that action. It sends the same key with every try. The server saves each key with its result. If the key comes again, the server returns the saved result and does not charge again. A new payment gets a new key.

Remember

• Always set a timeout.
• Retry brief failures, not errors that will repeat.
• Back off, and add jitter.
• Retry only what is idempotent, or send an idempotency key.