O'Reilly Site Reliability Engineering Chapter

Learn all you need to know about email best practices, deliverability, and tools with email whitepapers and ebooks.

Issue link: https://hub.dyn.com/i/961134

Contents of this Issue


Page 10 of 14

To really solve the problem of frontend load balancing, this initial level of DNS load balancing should be followed by a level that takes advantage of virtual IP addresses. Load Balancing at the Virtual IP Address Virtual IP addresses (VIPs) are not assigned to any particular network interface. Instead, they are usually shared across many devices. However, from the user's per‐ spective, the VIP remains a single, regular IP address. In theory, this practice allows us to hide implementation details (such as the number of machines behind a particu‐ lar VIP) and facilitates maintenance, because we can schedule upgrades or add more machines to the pool without the user knowing. In practice, the most important part of VIP implementation is a device called the net‐ work load balancer. The balancer receives packets and forwards them to one of the machines behind the VIP. These backends can then further process the request. There are several possible approaches the balancer can take in deciding which back‐ end should receive the request. The first (and perhaps most intuitive) approach is to always prefer the least loaded backend. In theory, this approach should result in the best end-user experience because requests are always routed to the least busy machine. Unfortunately, this logic breaks down quickly in the case of stateful proto‐ cols, which must use the same backend for the duration of a request. This require‐ ment means that the balancer must keep track of all connections sent through it in order to make sure that all subsequent packets are sent to the correct backend. The alternative is to use some parts of a packet to create a connection ID (possibly using a hash function and some information from the packet), and to use the connection ID to select a backend. For example, the connection ID could be expressed as: id(packet) mod N where id is a function that takes packet as an input and produces a connection ID, and N is the number of configured backends. This avoids storing state, and all packets belonging to a single connection are always forwarded to the same backend. Success? Not quite yet. What happens if one backend fails and needs to be removed from the backend list? Suddenly N becomes N-1 and then, id(packet) mod N becomes id(packet) mod N-1. Almost every packet sud‐ denly maps to a different backend! If backends don't share any state between them‐ selves, this remapping forces a reset of almost all of the existing connections. This scenario is definitely not the best user experience, even if such events are infrequent. Fortunately, there is an alternate solution that doesn't require keeping the state of every connection in memory, but won't force all connections to reset when a single machine goes down: consistent hashing. Proposed in 1997, consistent hashing describes a way to provide a mapping algorithm that remains relatively stable even Load Balancing at the Virtual IP Address | 5

Articles in this issue

view archives of eBooks - O'Reilly Site Reliability Engineering Chapter