Patterns

35 items

System Design Pattern

Reliabilitycircuit-breakerfault-tolerancecascade-failurefail-fastrecoveryintermediate

Circuit Breaker Pattern

Preventing cascade failures in distributed systems

Used in: Microservices, API Gateways, Service Mesh|20 min read

Summary

Circuit Breaker is a resilience pattern that prevents cascade failures by automatically stopping requests to failing services. Like an electrical circuit breaker, it has three states: Closed (normal operation), Open (failing fast without making requests), and Half-Open (testing if service recovered). When failure rate exceeds a threshold, the circuit "trips" open, giving the downstream service time to recover while failing fast to protect the caller. This pattern is essential for production systems - Netflix's Hystrix library popularized it and prevented countless outages.

Key Takeaways

Three States Prevent Cascade Failures

Closed state operates normally, Open state fails fast without calling the service, Half-Open state tests recovery with limited requests. This state machine prevents a failing service from bringing down its callers.

Automatic Recovery Testing

After a timeout period (e.g., 30-60s), the circuit automatically enters Half-Open state to test if the service recovered. If test requests succeed, it closes; if they fail, it opens again.

Fail Fast Protects Resources

When open, requests fail immediately without consuming threads, connections, or timeouts. This prevents thread pool exhaustion and allows the system to maintain capacity for other operations.

In microservices architectures, services depend on each other. When one service fails or becomes slow, it can bring down all services that depend on it - a cascade failure.

The cascade failure scenario:

Service A calls Service B, which calls Service C
Service C starts failing (database overload, memory leak)
Service B waits for responses, consuming threads
Service B thread pool exhausts, stops responding
Service A exhausts its thread pool waiting for Service B
The entire system collapses like dominoes

Circuit Breaker State Machine

Netflix Origin

Netflix built Hystrix after cascade failures brought down streaming. The library prevented countless outages and influenced all modern circuit breaker implementations including Resilience4j, Polly, and Spring Cloud.

Summary

Key Takeaways

Three States Prevent Cascade Failures

Automatic Recovery Testing

After a timeout period (e.g., 30-60s), the circuit automatically enters Half-Open state to test if the service recovered. If test requests succeed, it closes; if they fail, it opens again.

Fail Fast Protects Resources

When open, requests fail immediately without consuming threads, connections, or timeouts. This prevents thread pool exhaustion and allows the system to maintain capacity for other operations.

Sliding Window Metrics

Modern circuit breakers use sliding windows (count-based or time-based) to calculate failure rates. This makes them responsive to recent failures while being resilient to occasional errors.

Fallback Strategies

When the circuit opens, return cached data, default values, or degraded functionality rather than propagating errors. This provides graceful degradation instead of complete failure.

Observability is Critical

Circuit state changes are important signals. Emit metrics and alerts when circuits open/close. Track failure rates, response times, and circuit state per service endpoint.

Pattern Details

In microservices architectures, services depend on each other. When one service fails or becomes slow, it can bring down all services that depend on it - a cascade failure.

The cascade failure scenario:

Service A calls Service B, which calls Service C
Service C starts failing (database overload, memory leak)
Service B waits for responses, consuming threads
Service B thread pool exhausts, stops responding
Service A exhausts its thread pool waiting for Service B
The entire system collapses like dominoes

Circuit Breaker State Machine

Netflix Origin

Trade-offs

Aspect	Advantage	Disadvantage

Patterns

Horizontal Scaling Pattern

Retry with Backoff Pattern

Replication Pattern

Caching Strategies Pattern

Persistent Connections Pattern

Load Balancing Pattern

Fan-out Pattern

Fan-in Pattern