A load balancer's job is to sit in front of a fleet of servers and decide, for each incoming request, which one actually handles it. Which algorithm it uses matters, because the wrong one for the workload either leaves capacity sitting idle or actively concentrates load on the servers least able to handle more of it.
| Algorithm | How it decides | Good fit | Weak point |
|---|---|---|---|
| Round robin | Cycles through servers in fixed order, one request each | Requests are roughly uniform in cost | Ignores actual server load — a server stuck on a slow request still gets the next one in rotation |
| Least connections | Sends the request to whichever server currently has the fewest open connections | Request costs vary a lot (some fast, some slow) | Slightly more bookkeeping than round robin, but usually worth it |
| Weighted round robin / least connections | Same as above, but bigger servers get proportionally more requests | A fleet with mixed hardware sizes | Weights have to be kept honest as hardware changes |
| Consistent hashing | Maps both servers and requests onto a hash ring; a request goes to the nearest server clockwise on the ring | You need the same request (or key) to reliably land on the same server | More complex to implement correctly than the others |
Round robin and least-connections are both fine defaults when any server can serve any request equally well and you don't care which one gets it. Consistent hashing solves a different problem entirely: you need repeat requests for the same key to keep landing on the same server — most commonly because that server is holding a cache or a piece of sharded data for that key, and sending the request anywhere else would be a cache miss or a wrong-shard lookup.
The naive way to route a key to a server is hash(key) % number_of_servers. It works, right up until the number of servers changes — add or remove even one server, and number_of_servers changes, which means the result of % number_of_servers changes for almost every key, not just the ones that logically needed to move. Practically every cache entry now maps to a different server than it used to, which reads as a near-total cache wipe at the exact moment you were trying to add capacity.
Consistent hashing fixes this by hashing servers onto positions on a fixed, circular hash ring (typically 0 to 2³²-1, wrapping back to 0), and hashing each key onto that same ring. A key is owned by whichever server's position is the first one reached going clockwise from the key's position:
Hash ring (positions increase clockwise, wrapping at the top):
Server A (pos 10)
/ \
Server D (pos 300) Server B (pos 90)
\ /
Server C (pos 200)
key "user:42" hashes to position 45 -> owned by Server B (next clockwise from 45)
key "user:7" hashes to position 250 -> owned by Server D (next clockwise from 250)
Now add a new server E at position 70. Only the keys that fall between the previous server going clockwise and E's new position get remapped — everything from position 11 through 70, which used to belong to Server B, now belongs to E instead. Every other key on the ring — everything owned by C, D, and most of A and B's original ranges — is completely untouched. Removing a server works the same way in reverse: its keys simply fall to whichever server is next clockwise, and nothing else moves.
That's the entire value proposition in one sentence: consistent hashing turns "resize the cluster" from an event that reshuffles nearly every key into one that reshuffles only the keys that specifically belonged to the server that joined or left. Real implementations also hash each physical server to several points on the ring ("virtual nodes") rather than just one, specifically to avoid one server's arc of the ring being disproportionately large or small purely by hash-function luck — without virtual nodes, an unlucky physical server could end up owning a wildly uneven share of the keyspace.
The other axis is which layer of the network stack the load balancer is actually making decisions at:
/api/orders/* to the Orders service and /api/users/* to the Users service, or route based on a session cookie for sticky sessions, none of which an L4 balancer can see at all.L4 is faster and cheaper because there's less to inspect per packet; L7 is what you reach for the moment routing needs to depend on anything about the request beyond "which IP and port is this."