SystemExpertsDesign Walkthrough
Problem Statement
The Question: Design a real-time stock price viewer that shows live prices to millions of users worldwide.
This system is essential for: - Trading platforms (Robinhood, E*TRADE) - Users need current prices to make decisions - Financial news (Bloomberg, Yahoo Finance) - Displaying market data - Portfolio trackers - Showing real-time portfolio value - Crypto exchanges (Coinbase) - 24/7 price updates
What to say first
Before I design, let me clarify the requirements. I want to understand the scale, latency requirements, and whether we need to handle trading or just viewing.
Hidden requirements interviewers test: - Do you understand the fan-out challenge? - Can you design for real-time at scale? - Do you know WebSocket connection management? - Can you handle hot keys (popular symbols)?
Clarifying Questions
Ask these questions to demonstrate senior thinking. Each answer shapes your architecture.
Question 1: Scale
How many concurrent users? How many symbols do we track? What is the update frequency from exchanges?
Why this matters: Determines connection infrastructure and fan-out strategy. Typical answer: 10M concurrent users, 50K symbols, 100K updates/sec from exchanges Architecture impact: Need distributed WebSocket servers and hierarchical fan-out
Question 2: Latency Requirements
What is the acceptable latency from exchange tick to user screen? Is this for trading or just viewing?
Why this matters: Trading requires sub-10ms; viewing can tolerate 100-500ms. Typical answer: Viewing platform, sub-100ms acceptable Architecture impact: Can batch updates, do not need exchange co-location
Question 3: Subscription Model
Do users subscribe to specific symbols or see everything? Can one user watch 100 symbols?
Why this matters: Determines connection multiplexing strategy. Typical answer: Users subscribe to watchlist (10-50 symbols typically) Architecture impact: Connection-level subscription management, not global broadcast
Question 4: Data Requirements
Just current price? Or also bid/ask, volume, chart data, order book?
Why this matters: Order book is much higher frequency than last trade price. Typical answer: Price, bid/ask, volume, percent change Architecture impact: Moderate data per update (~100 bytes)
Stating assumptions
I will assume: 10M concurrent users, 50K symbols, 100K updates/sec from exchanges, sub-100ms latency goal, subscription-based model with 10-50 symbols per user.
The Hard Part
Say this out loud
The hard part here is the fan-out problem. When AAPL price updates, we need to push that update to 5 million subscribers within 100ms. No single server can maintain 5M WebSocket connections.
Why this is genuinely hard:
- 1.Massive Fan-out: Popular symbols like AAPL, TSLA, SPY have millions of watchers. One price tick becomes millions of pushes.
- 2.Connection Limits: A single server can handle maybe 100K WebSocket connections. 10M users need 100+ servers.
- 3.Hot Symbols: Distribution is highly skewed. Top 100 symbols might have 90% of subscribers.
- 4.Ordering Guarantees: Users must see prices in order. Cannot show 151 then 150.
- 5.Global Distribution: Users are worldwide. Cannot have everyone connect to US servers.
Common mistake
Candidates often propose a simple pub/sub where the price service publishes and all WebSocket servers subscribe. This creates thundering herd - 100K updates/sec x 100 servers = 10M messages/sec on the message bus.
The fundamental insight:
We need hierarchical fan-out:
Exchange -> Aggregator -> Regional Hubs -> Edge Servers -> Users
1 1 10 1000 10MEach layer amplifies to the next. No single component handles the full fan-out.
Scale and Access Patterns
Let me estimate the scale and understand access patterns.
| Dimension | Value | Impact |
|---|---|---|
| Concurrent users | 10,000,000 | Need 100+ WebSocket servers (100K conn each) |
| Total symbols | 50,000 | Manageable, can fit in memory |
What to say
The key insight is the skewed distribution. Top 100 symbols have 90% of subscribers. We need special handling for these hot symbols.
Access Pattern Analysis:
- Read-heavy at edge: Users mostly receive updates, rarely send - Write-heavy at ingestion: 100K updates/sec coming in from exchanges - Skewed popularity: Power law distribution of symbol popularity - Bursty traffic: Market open/close, earnings announcements cause spikes - Time-based patterns: US market hours have 10x more traffic
Fan-out calculation:
- AAPL has 5M subscribers across 50 edge servers
- Each edge server has 100K AAPL subscribersHigh-Level Architecture
Let me design the architecture with hierarchical fan-out.
What to say
I will design a hierarchical fan-out system. Updates flow from exchanges through aggregators to regional hubs to edge servers to users. Each layer handles a bounded fan-out.
Stock Price Viewer Architecture
Component Responsibilities:
1. Aggregator Layer - Connects to exchange feeds (NYSE, NASDAQ, crypto) - Normalizes data format - Deduplicates (same stock on multiple exchanges) - Adds server timestamp
2. Regional Hubs - One per major region (US, EU, Asia) - Receives all updates from aggregators - Maintains current price for all symbols - Pushes to edge servers in that region
3. Edge Servers - Maintain WebSocket connections to users - Subscribe to regional hub for symbols their users watch - Batch updates before pushing to users - Handle connection lifecycle
Real-world reference
Bloomberg Terminal uses a similar hierarchical architecture. Yahoo Finance uses CDN edge nodes with WebSocket support. Robinhood batches updates on 100ms intervals.
Data Model and Storage
The data model is simple but the storage strategy is nuanced.
What to say
We use Redis for current prices (hot data) and a time-series database for historical data. The key insight is that current prices are read millions of times but written only when they change.
Current Price (Redis Hash):
Key: price:AAPL
Fields:Storage Strategy:
| Data | Storage | Why | |------|---------|-----| | Current prices | Redis Cluster | Sub-ms reads, 50K keys fits in memory | | Subscription registry | Redis | Fast lookup of which edge servers need updates | | Historical prices | TimescaleDB/InfluxDB | Time-series optimized, chart queries | | User connections | In-memory on edge | No persistence needed, rebuilt on reconnect |
class PriceAggregator:
def __init__(self):
self.redis = RedisCluster()Sequence Numbers:
Every update has a sequence number to ensure ordering:
class EdgeServer:
def __init__(self):
self.last_sequence = {} # symbol -> last seen sequenceWebSocket Connection Management
Managing millions of WebSocket connections is a core challenge.
Say this out loud
WebSocket connection management at scale requires careful attention to memory, file descriptors, and graceful handling of disconnections.
| Challenge | Solution | Implementation |
|---|---|---|
| Connection limits | Multiple edge servers | 100K connections per server, load balance by user |
| Memory per connection | Minimal state | Only store subscription list and sequence numbers |
| Heartbeats | Client-initiated pings | Server responds to pings, times out after 30s |
| Reconnection storms | Exponential backoff | Client waits 1s, 2s, 4s, 8s... on reconnect |
| Graceful shutdown | Connection draining | Stop accepting new, wait for existing to migrate |
class EdgeWebSocketServer:
def __init__(self):
self.connections = {} # conn_id -> WebSocketBatching Updates:
To reduce CPU and network overhead, batch updates:
class BatchedBroadcaster:
def __init__(self, batch_interval_ms=100):
self.batch_interval = batch_interval_ms / 1000Why batching works
A stock might update 100 times per second, but users cannot perceive changes faster than 10/sec. Batching to 100ms intervals reduces fan-out by 10x with no visible impact.
Consistency and Invariants
System Invariants
Users must never see prices go backwards. If AAPL shows 150 then 151, no user should ever see 151 then 150. This requires sequence numbers and client-side ordering.
Consistency Model:
We use causal consistency - updates for the same symbol are seen in order, but different symbols may be slightly out of sync.
| Guarantee | How We Achieve It | Why It Matters |
|---|---|---|
| Same-symbol ordering | Sequence numbers per symbol | Prices cannot go backwards |
| Bounded staleness | 100ms batch window max | Users see reasonably current prices |
| No duplicates | Sequence deduplication | Count/sum calculations are correct |
| Eventual delivery | Retries with backoff | Users eventually see all updates |
class StockPriceClient {
constructor() {
this.lastSequence = new Map(); // symbol -> sequenceBusiness impact mapping
If a user briefly sees stale price (100ms old), minor inconvenience. If they see prices going backwards, they lose trust in the platform. We optimize for ordering over freshness.
Handling Network Partitions:
If an edge server loses connection to the hub:
- 1.Detect: No heartbeat from hub for 5 seconds 2. Inform users: Send special message indicating stale data 3. Reconnect: Try other hub replicas 4. Resync: Request current prices for all subscribed symbols
Failure Modes and Resilience
Proactively discuss failures
Let me walk through failure modes. For a financial data system, reliability is critical.
| Failure | Impact | Mitigation | Recovery |
|---|---|---|---|
| Edge server crash | 100K users disconnected | Users reconnect to other servers | Auto-resubscribe on connect |
| Regional hub down | Region gets stale data | Failover to backup hub | Automatic with health checks |
Graceful Degradation:
class EdgeServerHealth:
HEALTHY = "healthy" # All systems normal
DEGRADED = "degraded" # Hub connection issuesClient Reconnection Strategy:
class StockWebSocket {
constructor(url) {
this.url = url;Why jitter matters
Without jitter, if an edge server crashes, all 100K clients try to reconnect at exactly 1s, 2s, 4s... This creates thundering herd on surviving servers. Jitter spreads the load.
Evolution and Scaling
What to say
This design handles 10M concurrent users. Let me discuss evolution for 100M users and additional features like personalized alerts.
Evolution Path:
Stage 1: Basic Viewer (10M users) - Hierarchical fan-out as designed - Regional deployment (US, EU, Asia) - 100ms batching
Stage 2: Enhanced Features (50M users) - Price alerts (notify when AAPL hits $150) - Portfolio value calculation (sum of holdings) - Historical chart data streaming
Stage 3: Trading Integration (100M users) - Lower latency path for traders (10ms) - Order book depth (Level 2 data) - Direct exchange connectivity
Enhanced Architecture with Alerts
Scaling Strategies:
| Scale Challenge | Current Design | At 10x Scale |
|---|---|---|
| More users | Add edge servers | Add more regions, edge CDN |
| More symbols | All in memory | Shard by symbol prefix |
| Lower latency | 100ms batching | 10ms batching, UDP multicast |
| More data | Price only | Dedicated streams per data type |
| Global reach | 3 regions | Edge compute at CDN POPs |
Alternative approach
For ultra-low-latency trading (sub-1ms), we would use UDP multicast from exchange directly to trading servers, bypassing the entire web infrastructure. This is what HFT firms do.
What I would do differently for...
Trading platform (not just viewing): - Co-locate servers at exchange data centers - Use kernel bypass networking (DPDK) - UDP multicast instead of TCP/WebSocket - Sub-millisecond latency focus
Mobile app with battery constraints: - Longer batch intervals (500ms-1s) - Smart throttling based on app state (foreground/background) - Push notifications for significant moves instead of streaming
Crypto (24/7, global): - No market hours, always-on architecture - Multiple exchange aggregation (price varies by exchange) - Higher update frequency (crypto is more volatile)