Patterns

35 items

System Design Pattern

Rate Managementrate-limitingtoken-bucketsliding-windowthrottlingapi-protectionintermediate

Rate Limiting Pattern

Controlling request rates to protect resources

Used in: API Gateway, Redis, Load Balancer|20 min read

Summary

Rate limiting controls how many requests a client can make in a given time window, protecting systems from abuse, ensuring fair usage, and preventing cascading failures. Common algorithms include Token Bucket (allows bursts), Leaky Bucket (smooth rate), Fixed Window (simple), and Sliding Window (accurate). Rate limits can be applied per user, API key, IP address, or globally. Every public API uses rate limiting - Twitter limits 300 tweets/3hrs, GitHub limits 5000 requests/hour. Essential for both protecting your system and providing fair access.

Key Takeaways

Multiple Dimensions

Rate limit by user, API key, IP, endpoint, or combinations. Different limits for different tiers (free vs paid). Protect both per-user and globally.

Token Bucket for Bursts

Bucket fills with tokens at steady rate. Each request consumes token. Allows short bursts up to bucket size. Good for API rate limiting.

Sliding Window for Accuracy

Fixed windows have boundary issues (double rate at window boundary). Sliding window smooths this but requires more state.

Token Bucket: - Bucket holds N tokens (burst capacity) - Tokens added at rate R per second - Request takes 1 token - No token = rejected - Allows bursts up to N

Leaky Bucket: - Requests enter bucket - Processed at fixed rate - Overflow rejected - Smooth output rate

Fixed Window: - Count requests in fixed intervals (per minute) - Reset count at interval boundary - Simple but has boundary spike issue

Sliding Window: - Weighted count across windows - Smoother than fixed window - More accurate, more state

Summary

Key Takeaways

Multiple Dimensions

Rate limit by user, API key, IP, endpoint, or combinations. Different limits for different tiers (free vs paid). Protect both per-user and globally.

Token Bucket for Bursts

Bucket fills with tokens at steady rate. Each request consumes token. Allows short bursts up to bucket size. Good for API rate limiting.

Sliding Window for Accuracy

Fixed windows have boundary issues (double rate at window boundary). Sliding window smooths this but requires more state.

Response Headers Convention

Return X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset. Clients can adjust behavior. Standard practice.

Distributed Rate Limiting

Multiple servers need shared state (Redis). Local-only limits can be bypassed. Synchronization adds latency.

Graceful Degradation

When limit exceeded, return 429 Too Many Requests. Include Retry-After header. Don't just drop requests silently.

Pattern Details

Token Bucket: - Bucket holds N tokens (burst capacity) - Tokens added at rate R per second - Request takes 1 token - No token = rejected - Allows bursts up to N

Leaky Bucket: - Requests enter bucket - Processed at fixed rate - Overflow rejected - Smooth output rate

Fixed Window: - Count requests in fixed intervals (per minute) - Reset count at interval boundary - Simple but has boundary spike issue

Sliding Window: - Weighted count across windows - Smoother than fixed window - More accurate, more state

Trade-offs

Aspect	Advantage	Disadvantage

Patterns

Horizontal Scaling Pattern

Retry with Backoff Pattern

Queue-based Load Leveling Pattern

Replication Pattern

Caching Strategies Pattern

Fan-out Pattern

Fan-in Pattern

Persistent Connections Pattern

Load Balancing Pattern

Circuit Breaker Pattern

Bloom Filters Pattern

Time-Series Storage Pattern

Bulkhead Pattern

Batch Processing Pattern

Write-Ahead Log Pattern

API Gateway Pattern

Backend for Frontend Pattern

Sidecar Pattern

Idempotency Pattern