Patterns

35 items

System Design Pattern

Communicationpub-submessagingdecouplingtopicssubscribersintermediate

Pub/Sub Pattern

Decoupled message distribution through topics

Used in: Kafka, Redis Pub/Sub, Google Pub/Sub|25 min read

Summary

The Publish/Subscribe pattern decouples message producers (publishers) from consumers (subscribers) through an intermediary message broker. Publishers send messages to named topics without knowing who will receive them, while subscribers express interest in one or more topics and receive only relevant messages. This decoupling enables scalable, flexible communication in distributed systems and is the foundation of event-driven architectures.

Key Takeaways

Decoupling Through Topics

Publishers and subscribers never communicate directly. Instead, they interact through named topics managed by a message broker. This loose coupling is fundamental to microservices architectures.

Dynamic Discovery and Fan-out

Subscribers can dynamically join or leave topics at runtime without affecting publishers or other subscribers. When a message is published, the broker automatically fans it out to all active subscribers.

At-Least-Once vs Exactly-Once Semantics

Most pub/sub systems guarantee at-least-once delivery: messages may be delivered multiple times during failures. This requires subscribers to implement idempotent message handlers.

In traditional point-to-point communication, services make direct requests to each other. This creates tight coupling: availability coupling, temporal coupling, knowledge coupling, and scalability limitations.

Pub/Sub Architecture

Summary

Key Takeaways

Decoupling Through Topics

Publishers and subscribers never communicate directly. Instead, they interact through named topics managed by a message broker. This loose coupling is fundamental to microservices architectures.

Dynamic Discovery and Fan-out

At-Least-Once vs Exactly-Once Semantics

Most pub/sub systems guarantee at-least-once delivery: messages may be delivered multiple times during failures. This requires subscribers to implement idempotent message handlers.

Message Ordering Guarantees

Global message ordering across all publishers is impossible in distributed pub/sub systems. Most systems guarantee ordering per partition or per publisher, not globally.

Push vs Pull Consumption Models

Pub/sub systems use either push (broker sends to subscribers) or pull (subscribers fetch from broker) models. Choose based on latency and backpressure requirements.

Topic Design and Granularity

Topic granularity significantly impacts system performance and flexibility. Fine-grained topics enable precise subscriptions but increase overhead.

Pattern Details

Pub/Sub Architecture

Trade-offs

Aspect	Advantage	Disadvantage

Patterns

Horizontal Scaling Pattern

Retry with Backoff Pattern

Queue-based Load Leveling Pattern

Replication Pattern

Caching Strategies Pattern

Fan-out Pattern

Fan-in Pattern

Persistent Connections Pattern

Load Balancing Pattern

Circuit Breaker Pattern

Bloom Filters Pattern

Time-Series Storage Pattern

Bulkhead Pattern

Batch Processing Pattern

Write-Ahead Log Pattern

API Gateway Pattern

Backend for Frontend Pattern

Sidecar Pattern

Idempotency Pattern

Rate Limiting Pattern

Backpressure Pattern

Pub/Sub Pattern

Eventual Consistency Pattern

Sharding Pattern

Conflict Resolution Pattern

Strong Consistency Pattern

Leader Election Pattern

Consensus Protocols Pattern

Stream Processing Pattern

Change Data Capture Pattern

Distributed Locking Pattern

Two-Phase Commit Pattern

LSM Trees Pattern

Event Sourcing Pattern

CQRS Pattern

Pub/Sub Pattern

Summary

Key Takeaways

Decoupling Through Topics

Dynamic Discovery and Fan-out

At-Least-Once vs Exactly-Once Semantics

The Problem: Tightly Coupled Communication

Pub/Sub Architecture

Premium Content

Pub/Sub Pattern

Summary

Key Takeaways

Decoupling Through Topics

Dynamic Discovery and Fan-out

At-Least-Once vs Exactly-Once Semantics

The Problem: Tightly Coupled Communication

Pub/Sub Architecture

Premium Content