Patterns

35 items

System Design Pattern

Scalabilityload-balancinground-robinconsistent-hashinghealth-checksl4-l7intermediate

Load Balancing Pattern

Distributing traffic across multiple servers

Used in: API Gateway, Nginx, HAProxy|20 min read

Summary

Load balancing distributes incoming traffic across multiple servers to prevent any single server from becoming overwhelmed. Load balancers use algorithms like Round Robin (simple rotation), Least Connections (route to least busy), Weighted (proportional to capacity), and IP Hash (sticky sessions). Modern load balancers also perform health checks, SSL termination, and can route based on content. This pattern is fundamental to horizontal scaling - without load balancing, you cannot utilize multiple servers effectively. Every production system uses load balancing, from simple DNS round-robin to sophisticated L7 application load balancers.

Key Takeaways

Even Distribution is Key

Goal is to spread load evenly so no server is overwhelmed while others are idle. Uneven distribution wastes capacity and creates bottlenecks.

Health Checks Prevent Failures

Load balancer must detect unhealthy servers and stop sending traffic. Without health checks, requests go to dead servers and fail.

L4 vs L7 Load Balancing

L4 (transport) routes based on IP/port - fast, simple. L7 (application) routes based on HTTP content - flexible, can route by URL, headers, cookies.

Round Robin: Rotate through servers sequentially - Simple, fair distribution - Does not account for server capacity or current load

Least Connections: Route to server with fewest active connections - Better for varying request durations - Requires connection tracking

Weighted: Route proportionally to server capacity - Server with weight 2 gets 2x traffic of weight 1 - Good for mixed-capacity fleet

IP Hash: Hash client IP to determine server - Same client always goes to same server - Provides sticky sessions without cookies

Least Response Time: Route to fastest responding server - Optimizes for latency - Requires latency monitoring

Load Balancing Algorithms

Summary

Key Takeaways

Even Distribution is Key

Goal is to spread load evenly so no server is overwhelmed while others are idle. Uneven distribution wastes capacity and creates bottlenecks.

Health Checks Prevent Failures

Load balancer must detect unhealthy servers and stop sending traffic. Without health checks, requests go to dead servers and fail.

L4 vs L7 Load Balancing

L4 (transport) routes based on IP/port - fast, simple. L7 (application) routes based on HTTP content - flexible, can route by URL, headers, cookies.

Sticky Sessions Trade-offs

Sticky sessions route same client to same server - required for stateful apps. But reduces load distribution and complicates failover.

SSL Termination

Load balancer can terminate SSL, sending unencrypted traffic to backends. Offloads crypto from servers, simplifies certificate management.

Geographic Load Balancing

DNS-based load balancing routes users to nearest datacenter. Reduces latency, enables disaster recovery, complies with data locality requirements.

Pattern Details

Round Robin: Rotate through servers sequentially - Simple, fair distribution - Does not account for server capacity or current load

Least Connections: Route to server with fewest active connections - Better for varying request durations - Requires connection tracking

Weighted: Route proportionally to server capacity - Server with weight 2 gets 2x traffic of weight 1 - Good for mixed-capacity fleet

IP Hash: Hash client IP to determine server - Same client always goes to same server - Provides sticky sessions without cookies

Least Response Time: Route to fastest responding server - Optimizes for latency - Requires latency monitoring

Load Balancing Algorithms

Trade-offs

Aspect	Advantage	Disadvantage

Patterns

Horizontal Scaling Pattern

Retry with Backoff Pattern

Replication Pattern

Caching Strategies Pattern

Persistent Connections Pattern