Patterns

35 items

System Design Pattern

Coordinationleader-electionconsensuscoordinationzookeepersingle-leaderadvanced

Leader Election Pattern

Selecting a single coordinator among distributed nodes

Used in: ZooKeeper, etcd, Consul|25 min read

Summary

Leader election is a coordination pattern where distributed nodes agree on a single leader to coordinate activities. Only the leader performs certain operations (writes, scheduling, coordination) while followers remain on standby. If the leader fails, remaining nodes elect a new leader. This pattern prevents split-brain scenarios, ensures single-writer semantics, and simplifies distributed coordination. Implementations include Raft consensus, ZooKeeper recipes, and etcd. Leader election is fundamental to databases (primary/replica), message brokers (partition leaders), and orchestration systems (scheduler).

Key Takeaways

Single Coordinator Simplifies Logic

With one leader, no need for distributed consensus on every operation. Leader makes decisions, followers accept. Dramatically simplifies system design.

Failure Detection is Critical

Must quickly detect leader failure to minimize downtime. But false positives cause unnecessary elections. Balance sensitivity vs stability.

Elections Must Be Safe

Can never have two leaders simultaneously (split-brain). Use fencing tokens, epoch numbers, or lease expiration to prevent stale leaders.

Problems without leader election:

Duplicate work: Multiple nodes do same task
Conflicting decisions: Nodes make different choices
Split-brain: Two primaries accept writes, data diverges

With leader election: - One node coordinates, others follow - Clear authority for decisions - Automatic failover when leader fails

Leader Election Flow

Summary

Key Takeaways

Single Coordinator Simplifies Logic

With one leader, no need for distributed consensus on every operation. Leader makes decisions, followers accept. Dramatically simplifies system design.

Failure Detection is Critical

Must quickly detect leader failure to minimize downtime. But false positives cause unnecessary elections. Balance sensitivity vs stability.

Elections Must Be Safe

Can never have two leaders simultaneously (split-brain). Use fencing tokens, epoch numbers, or lease expiration to prevent stale leaders.

Lease-Based vs Consensus-Based

Leases: Leader holds time-limited lock, must renew. Simple but clock-dependent. Consensus: Nodes vote, majority wins. Complex but clock-independent.

Followers Must Reject Stale Leaders

After election, old leader may not know it's been replaced. Include term/epoch in all requests. Followers reject requests from old terms.

Leader Election is Building Block

Used by Kafka (partition leader), Elasticsearch (master), Redis Sentinel, Kubernetes (scheduler). Understanding it unlocks understanding many systems.

Pattern Details

Problems without leader election:

Duplicate work: Multiple nodes do same task
Conflicting decisions: Nodes make different choices
Split-brain: Two primaries accept writes, data diverges

With leader election: - One node coordinates, others follow - Clear authority for decisions - Automatic failover when leader fails

Leader Election Flow

Trade-offs

Aspect	Advantage	Disadvantage

Patterns

Horizontal Scaling Pattern

Retry with Backoff Pattern

Replication Pattern

Caching Strategies Pattern

Persistent Connections Pattern

Load Balancing Pattern

Fan-out Pattern

Fan-in Pattern

Circuit Breaker Pattern

Eventual Consistency Pattern

Queue-based Load Leveling Pattern

Bloom Filters Pattern

Time-Series Storage Pattern

Bulkhead Pattern

Batch Processing Pattern

Write-Ahead Log Pattern

API Gateway Pattern

Backend for Frontend Pattern

Sidecar Pattern

Idempotency Pattern

Rate Limiting Pattern

Backpressure Pattern

Pub/Sub Pattern

Strong Consistency Pattern

Conflict Resolution Pattern