Scaling a web app

Latency and throughput

Latency is how long one request takes, like 200 ms. Throughput is how many requests the app handles per second. A server that handles one request at a time, 100 ms each, has a throughput of 10 requests per second.

Scaling up

Scaling up (vertical scaling) means a bigger machine, with more CPU and memory. It is simple, because the app does not change. But even the biggest machine has a limit, and it is still one machine. If it dies, the whole app is down. That makes it a single point of failure.

Scaling out

Scaling out (horizontal scaling) means adding more machines that share the work. Four servers that each handle 200 requests per second can handle 800 together. If one crashes, the others keep going. More servers raise throughput, but they do not make one request faster.

Load balancer

A load balancer sits in front of the servers and sends each request to one of them. A simple way to pick is round robin: A, then B, then C, then A again. It runs health checks, small tests that ask each server if it is up. If a server stops answering, it gets no more requests.

Stateless servers

If one server kept your cart in its memory, your next request could reach another server and find the cart empty. So a stateless server keeps no user data in its own memory between requests. Logins and carts live in a shared store, such as a database, that every server can read. Then any server can answer any request, and you can add or remove servers freely.

Remember

• Latency is time per request; throughput is requests per second.
• Scaling up is simple but has a limit and is one point of failure.
• Scaling out adds machines and survives a crash.
• A load balancer spreads requests and skips servers that are down.
• Keep user data in a shared store so any server can answer.