• Part 0: Entire picture
• Step 1 - Understand the problem and establish design scope
• Functional requirements
• Non-functional requirements
• Step 2 - Propose high-level design and get buy-in
• Step 3 - Design deep dive
• Step 4 - Wrap up
• Monitoring Metrics, Logging, Automation
• Testing

Part 0: Entire picture

Step 1 - Understand the problem and establish design scope

‣

Part 1: Single server setup

Functional requirements

Non-functional requirements

Estimate load on system, QUERY-PER-SECOND, and STORAGE requirements

• How many users does the product have?
• How many daily active users will we have?
• How many users would be concurrently using the service on average and at peak times?
• How many actions will each user do daily?
• What is the ratio of writes and reads?
• Ask about static files?
• How many transactions per second we could have? Estimate it by yourself.

‣

Estimations: QPS, peak QPS, storage, cache, number of servers

Step 2 - Propose high-level design and get buy-in

Should we include API endpoints and database schema here?

1. Let’s talk about API design. What kind of resources we have? Follow CRUD. INPUT/OUTPUT.
1. Entities (properties and their types)
2. Endpoints GET, POST, PATCH/PUT, DELETE.
3. Request headers, parameters. Response schema.
‣
Best practices of designing API
• API Version
• Pagination (offset limit)
• Order, filtering using query string.
‣
HTTP status codes

2. Database schema
3. Application server and database server are in the same machine. Let’s separate them. Build distributed system.
4. Scaling application. Performance = LB + Caching + MQ
1. How to reduce load on server? Move the database to another machine.
‣
Part 2: Separating database.

1. Now, we have a distributed system. Both the machines would be in the same network. Our backend server would interact with the database through its private IP. No other computer should be able to access the database.
2. Still issues:
• Single point of failure: If this machine dies, we will not be able to serve requests and our website goes down.
• Still high load
3. Vertical Scaling (CPU, memory)

Pros: simplicity (no code change), scale (solves problem of high load)

Cons: limited resources on a single machine, single point of failure, expensive.

4. Horizontal Scaling (multiple smaller servers + Load balancer)

Pros: distributed load, no single point of failure (we can have multiple LB), cheap

Cons: system should be stateless, management (health check)

1. Problem with Stateless. Solve it by separating state into another server. Distributed key value store.
‣
Part 3: Let’s introduce Load Balancer

‣
Part 4: Multiple data centers. geoDNS-routed. Round Robin DNS.
Mention that Bloomberg uses DNS load balancing. Round Robin DNS. And it uses AWS.

5. Message queues & Asynchronous processing
6. Caching
7. Decoupling the work
8. Preprocessing and cron job, serverless, celery.
5. Scaling database. Availability & scalability = replication + sharding.
1. Problems with database:
• might not be able to handle the high traffic
• is a single point of failure
• might have increased latency for both 'reads' and 'writes
2. How to solve?
3. But before let’s select between SQL or NoSQL?
‣

Should we use SQL or NoSQL? RDBMS = ACID, NoSQL = scaling.

The choice of database depend on non-functional requirements: structure of data, query pattern, scalability.

RDBMS: Structured data (we know all columns=attributes), ACID, row based db. Ex: user info, pay.

• ACID: all is changed of nothing, data is changed and always same, isolation=no affect to other transaction, durability. Ex: sending money to other account.

NoSQL: Horizontal scaling. Good for hierarchical data. More storage, because of denormalization. Fast queries through denormalization.

• DocumentDB, MongoDB. Ex: category in e-commerce. Because there might be different products in category. And product has different property.
• ColumnarDB: column based db. CassandraDB.

Caching: Redis

Blob storage: S3 + CDN. Ex: images, video.

Search: Text search engine, Elastic search. Ex: google maps, netflix movie search.

Time series database

Analytics: DWH, Hadoop.

1. Indexes on columns. More space, slower update but fast query.
4. Database replication — master/slave
‣
Part 5: Replication


Master-slave replication: Master will handle writes/updates/delete. Slaves will copy data from master. Read and write replicas.

Types: sync, async (peer-to-peer), multi-async (multi-master replication).

Pros:

• Better performance, write and read in different servers. We have more reads
• Reliability, disaster recovery.
• Availability, No single point of failure, if server fails we can still handle requests.

Cons:

• Master selection strategy. If master fails.
• Consistency models. Consistency over availability? Show bank, chat. (Strong consistency, weak, eventual consistency). If W + R > N, strong consistency is guaranteed (Usually N = 3, W = R = 2).
5. Database scaling

Vertical Scaling (CPU, memory)

Horizontal scaling (sharding — adding more servers), and distribute data using some strategy.

1. Cons: Resharding data, celebrity problem, join and de-normalization.
2. Consistent hashing solves the problem of scalability
• Cons: If some shard goes down we lose the data. Maybe we could replicate the shards.
6. Server caching (TTL=time to live. Ex: Example: Storing the product information against a product id)
7. CDN, and client caching.

Step 3 - Design deep dive

Talk with your interviewer which one he/she wants to dig into. Ex: For URL shortener, it is interesting to dive into the hash function design that converts a long URL to a short one. For a chat system, how to reduce latency and how to support online/offline status are two interesting topics.

• Rate limiting
• Preprocessing and cron job, serverless, celery.
• Caching: client cache, server cache, CDN. Time-to-live.
• Message queues & Asynchronous processing
• Decoupling the work
• Consisting hashing
• Polling and streaming

Step 4 - Wrap up

Monitoring Metrics, Logging, Automation

• Throughput
• Latency
• Memory
• Storage
• CI/CD - deployment

Testing

• Functional and non-functional testing