The Universal System Design Framework

CANDIDATE: Before I start, let me understand what we are building.

CORE FEATURES:
Me: What are the must-have features?
Interviewer: Posting tweets and seeing your feed.
Me: Should I include likes, retweets, DMs?
Interviewer: Focus on posting and feed for now.

USERS:
Me: How many users are we designing for?
Interviewer: Think Twitter scale - hundreds of millions.
Me: Global users or concentrated in one region?
Interviewer: Global.

SCALE:
Me: Any specific numbers I should target?
Interviewer: Let us say 500 million users, 200 million daily active.

SPECIAL REQUIREMENTS:
Me: How fast should the feed load?
Interviewer: Under 200 milliseconds.
Me: Is it okay if a tweet takes a few seconds to appear in feeds?
Interviewer: Yes, a small delay is fine.

SUMMARY:
- Core: Post tweets + View feed
- Scale: 500M users, 200M daily active
- Latency: Feed loads in under 200ms
- Eventual consistency is okay (small delays acceptable)

CORE APIS:

1. POST A TWEET
   POST /tweets
   Input:  { user_id, content, media_ids (optional) }
   Output: { tweet_id, created_at }
   
2. GET HOME FEED
   GET /feed?user_id=123&limit=20&cursor=xyz
   Input:  user_id, how many tweets, where to start
   Output: { tweets: [...], next_cursor }
   
3. FOLLOW A USER
   POST /follow
   Input:  { follower_id, followee_id }
   Output: { success: true }

4. UNFOLLOW A USER
   DELETE /follow
   Input:  { follower_id, followee_id }
   Output: { success: true }

WHAT THIS TELLS US:
- We need to store: tweets, users, who follows whom
- GET /feed is called way more than POST /tweets (read-heavy)
- Feed needs to be fast (called on every app open)
- The hard question: How do we build /feed efficiently?

GIVEN:
- 200 million daily active users
- Each user opens app 5 times per day
- Each user posts 0.5 tweets per day (on average)
- Each user follows 200 people

REQUESTS PER SECOND:

Feed loads (reads):
  200M users × 5 opens = 1 billion feed loads per day
  1 billion ÷ 100,000 seconds = 10,000 reads/second

Tweet posts (writes):
  200M users × 0.5 tweets = 100 million tweets per day
  100 million ÷ 100,000 seconds = 1,000 writes/second

Ratio: 10:1 read-heavy system

STORAGE:

Tweet size: 280 chars + metadata = ~500 bytes
Per day: 100M tweets × 500 bytes = 50 GB per day
Per year: 50 GB × 365 = 18 TB per year

PEAK:
Peak is usually 3-5x average
So plan for: 50,000 reads/second, 5,000 writes/second

WHAT THIS TELLS US:
✓ Read-heavy: caching will help a lot
✓ 10,000+ RPS: need multiple servers
✓ 18 TB/year: big but manageable, sharding may be needed later

THE QUESTION:
When a user opens their feed, how do we show tweets from people they follow?

OPTION A: REAL-TIME (Fan-out on Read)

When user opens feed:
1. Look up everyone they follow (200 people)
2. Get recent tweets from each person
3. Merge and sort by time
4. Return top 20 tweets

Pros:
- Always fresh data
- Simple to understand
- No extra storage

Cons:
- Slow! Must query 200 users every time
- Does not scale for users following 10,000 accounts
- Every feed open = heavy database load

OPTION B: PRE-COMPUTE (Fan-out on Write)

When someone posts a tweet:
1. Find all their followers (could be millions)
2. Add this tweet to each follower feed (stored in cache)

When user opens feed:
1. Just read their pre-built feed from cache
2. Super fast - one read operation

Pros:
- Feed loads instantly
- Light database load on read

Cons:
- Celebrity problem: user with 50M followers = 50M writes
- Uses more storage (duplicate tweets in many feeds)
- Small delay before tweet appears in feeds

OPTION C: HYBRID (What Twitter Actually Does)

- For normal users (< 10K followers): pre-compute
- For celebrities (> 10K followers): real-time
- When building feed: get pre-computed tweets + fetch celebrity tweets

This is usually the right answer for social feeds.

TWITTER:
- Users table: PostgreSQL (relational, need ACID)
- Tweets table: PostgreSQL (need to query by time, user)
- Follower graph: PostgreSQL (simple join) or Graph DB (if complex queries)
- Feed cache: Redis (fast reads, okay to lose on crash)
- Tweet search: Elasticsearch (full-text search)

UBER:
- Users, Drivers, Trips: PostgreSQL (transactional)
- Driver locations: Redis (ephemeral, changes every second)
- Location history: Cassandra (high write volume, time-series)

E-COMMERCE:
- Products, Orders, Users: PostgreSQL (ACID for payments)
- Shopping cart: Redis (ephemeral, fast)
- Product search: Elasticsearch (full-text, facets)
- Product catalog: PostgreSQL + CDN cache

CHAT APPLICATION:
- Users, Conversations: PostgreSQL
- Messages: Cassandra (high write volume, time-ordered)
- Online status: Redis (ephemeral, changes often)
- Message search: Elasticsearch

COMPONENTS:

1. API GATEWAY
   - Handles authentication (is this user logged in?)
   - Rate limiting (is this user sending too many requests?)
   - Routes requests to the right service

2. TWEET SERVICE
   - POST /tweets endpoint
   - Writes tweet to database
   - Sends message to queue: "new tweet posted"

3. FEED SERVICE  
   - GET /feed endpoint
   - Reads pre-built feed from Redis cache
   - If cache miss, builds feed from database

4. FANOUT SERVICE (Background Worker)
   - Listens for "new tweet posted" messages
   - For each follower, adds tweet to their feed in cache

5. USER SERVICE
   - Follow/unfollow endpoints
   - Updates the social graph

6. DATA STORES:
   - PostgreSQL: Users, Tweets, Followers (source of truth)
   - Redis: Feed cache (user_id -> list of tweet_ids)
   - S3: Images and videos (blob storage)
   - CDN: Serves images and videos fast, globally

def get_user(user_id):
    # Step 1: Check cache first
    user = cache.get(f"user:{user_id}")
    
    if user is not None:
        return user  # Cache hit! Fast path
    
    # Step 2: Cache miss - get from database
    user = database.query("SELECT * FROM users WHERE id = ?", user_id)
    
    # Step 3: Store in cache for next time (with expiry)
    cache.set(f"user:{user_id}", user, expire=3600)  # 1 hour
    
    return user

STRATEGY 1: HASH-BASED SHARDING
- shard = hash(user_id) % number_of_shards
- Pros: Even distribution
- Cons: Hard to add shards later, cross-shard queries hard

STRATEGY 2: RANGE-BASED SHARDING
- Shard 1: user_id 1-1,000,000
- Shard 2: user_id 1,000,001-2,000,000
- Pros: Easy range queries, easy to add shards
- Cons: Can get uneven (what if shard 1 has all active users?)

STRATEGY 3: DIRECTORY-BASED SHARDING
- Keep a lookup table: user_id -> shard
- Pros: Complete control, can move users between shards
- Cons: Lookup table is single point of failure

WHICH TO CHOOSE:
- Start with hash-based (simplest)
- Use consistent hashing to make adding shards easier
- Only use range if you need range queries

PROBLEM:
Celebrity with 50 million followers posts a tweet.
If we fan-out on write, that is 50 million cache updates.
This will take forever and overwhelm our workers.

SOLUTION: HYBRID APPROACH

1. Classify users:
   - Normal user: < 10,000 followers
   - Celebrity: >= 10,000 followers

2. For normal users (fan-out on write):
   - When they post, add tweet to all followers feeds
   - Works fine for 10,000 updates

3. For celebrities (fan-out on read):
   - When they post, store tweet but do NOT fan out
   - Mark their account as "celebrity"

4. When user loads feed:
   - Get pre-built feed from cache (tweets from normal users)
   - Find celebrities this user follows
   - Fetch recent tweets from those celebrities
   - Merge and sort
   - Return to user

5. Optimization:
   - Cache celebrity tweets aggressively (everyone reads them)
   - Only fetch celebrities when user scrolls past pre-built feed

def call_external_service(request):
    max_retries = 3
    base_delay = 1  # second
    
    for attempt in range(max_retries):
        try:
            return service.call(request)
        except TemporaryError:
            if attempt == max_retries - 1:
                raise  # Give up after max retries
            
            # Wait longer each time: 1s, 2s, 4s
            delay = base_delay * (2 ** attempt)
            # Add some randomness to avoid thundering herd
            delay = delay + random(0, delay * 0.1)
            sleep(delay)

PROBLEM:
User clicks "Pay" button twice (network was slow).
Without idempotency: User is charged twice!

SOLUTION:
Each request has a unique idempotency key.

def process_payment(idempotency_key, amount):
    # Check if we already processed this
    existing = database.get("payment:" + idempotency_key)
    
    if existing:
        return existing.result  # Return same result as before
    
    # Process the payment
    result = payment_provider.charge(amount)
    
    # Store the result with the key
    database.store("payment:" + idempotency_key, result)
    
    return result

Now if user clicks twice with same idempotency_key:
- First click: Processes payment, stores result
- Second click: Returns stored result, no double charge

NETFLIX EXAMPLE:
- Normal: Personalized recommendations for you
- Recommendation service down: Show popular movies instead
- Search service down: Show categories to browse
- Everything down: Show cached homepage

TWITTER EXAMPLE:
- Normal: Real-time feed with all features
- Feed service slow: Show cached version of feed
- Image service down: Show tweets without images
- Everything down: Show cached tweets, disable posting

PAYMENT EXAMPLE:
- Primary payment provider down: Try backup provider
- All providers down: Queue payment for later, tell user
- Never: Silently fail and lose the payment

□ STEP 1: CLARIFY REQUIREMENTS (5 min)
  □ What are the 2-3 core features? (MVP mindset)
  □ Who are the users? How many?
  □ What scale are we designing for?
  □ Any special requirements? (real-time, reliability)
  □ Summarize before moving on

□ STEP 2: DESIGN THE API (3 min)
  □ List core endpoints (3-5 max)
  □ For each: method, path, input, output
  □ Which endpoint is the hot path?
  □ Is it read-heavy or write-heavy?

□ STEP 3: QUICK MATH (3 min)
  □ Requests per second
  □ Storage needed
  □ Read:Write ratio
  □ Peak vs average
  □ What do these numbers tell us?

□ STEP 4: COMPUTE MODEL (3 min)
  □ For each API: real-time or pre-compute?
  □ Explain the tradeoff
  □ Make a decision and justify

□ STEP 5: DATABASE CHOICE (3 min)
  □ What databases do we need?
  □ Start with SQL, justify any NoSQL
  □ Explain what goes where

□ STEP 6: DRAW ARCHITECTURE (10 min)
  □ Start simple: LB → App → DB
  □ Add cache if read-heavy
  □ Add queue if need async processing
  □ Keep it to 5-8 boxes
  □ Explain each component

□ STEP 7: HANDLE SCALE (10 min)
  □ Identify the bottleneck
  □ Caching strategy
  □ Sharding if needed
  □ Hot partition handling

□ STEP 8: ADD RELIABILITY (5 min)
  □ What happens when X fails?
  □ Replication for durability
  □ Retries with backoff
  □ Idempotency for duplicates

□ WRAP UP (3 min)
  □ Summarize what you built
  □ State the main tradeoffs
  □ Suggest future improvements
  □ Ask: Any questions or areas to explore?

PROBLEM: Design a Food Delivery System (like DoorDash)

STEP 1: REQUIREMENTS
- Core: Browse restaurants, place order, track delivery
- Users: Customers, restaurants, drivers
- Scale: 1 million orders per day

STEP 2: API
- GET /restaurants?lat=X&lng=Y (browse nearby)
- POST /orders (place order)
- GET /orders/{id} (track status)
- PUT /drivers/{id}/location (driver updates position)

STEP 3: MATH
- 1M orders/day = ~10 orders/second
- Driver location updates: 50K drivers × every 5 seconds = 10K/second
- Read-heavy for browsing, write-heavy for locations

STEP 4: COMPUTE MODEL
- Restaurant list: Pre-compute (does not change often)
- Driver location: Real-time (changes constantly)
- Order matching: Real-time (need to match quickly)

STEP 5: DATABASE
- PostgreSQL: Users, Restaurants, Orders (need transactions)
- Redis: Driver locations (ephemeral, fast updates)
- Elasticsearch: Restaurant search (full-text, geo)

STEP 6: ARCHITECTURE
- API Gateway → Restaurant Service, Order Service, Driver Service
- Redis for real-time driver locations
- Message queue for order → driver matching

STEP 7: SCALE
- Cache restaurant data (does not change often)
- Shard orders by city/region
- Hot spot: Popular restaurants - cache their data

STEP 8: RELIABILITY
- Orders must not be lost (database + queue)
- Driver location can be approximate (no replication needed)
- Payment must be idempotent (no double charges)