Replication, sharding

image

Scaling: The Art of Distribution

When one server isn't enough, we move into Distributed Systems. This is where the "Not Easy" part really kicks in. You mentioned splitting or replicating; in the industry, we categorize these as:

Replication (High Availability)

You copy the same data across multiple nodes.

  • Leader-Follower: One node handles writes; others handle reads. Great for read-heavy apps (like Twitter).
  • Multi-Leader: Good for local latency (e.g., a server in London and one in NYC), but creates massive "conflict resolution" headaches.

Partitioning / Sharding (Scalability)

You split the data into different buckets.

  • Horizontal Sharding: User IDs 1–1000 go to Server A; 1001–2000 go to Server B.
  • The Challenge: What happens if you need to run a report that joins data from both shards? It becomes an expensive network operation.

Replication

Database replication is the process of copying data from one database to another, in order to ensure that the data is available on multiple servers.

Databases are stateful systems. Redundant copies need to stay in sync so that they represent the same state. We can do that by replicating the changes to the other databases.

Types of replication

  • Synchronous replication = copying data in real-time to all replicas. Changes are propagated to all the database nodes before considering the transaction successful.

      • Pros:

    1. Data consistency in all databases. Used when data in all databases should be up-to-date.

      2. Cons:

    2. Slower because ex. WRITE operation should be done everywhere before returning success.
    3. The database might be unavailable while the replication is happening.
    4. What will happen if one of the databases fail?
  • Async replication = copying data at intervals. Changes are propagated asynchronously after the transaction has been considered successful.

      • Pros:

    1. The database would be available during that time.
    2. Used when the data on the target database does not need to be in real-time sync with the data on the source database.
    3. We can replicate when the network latency between the source and target databases is high.

      4. Cons:

    4. The system might be inconsistent while the replication is happening. CAP theorem.

      5. Example: Imagine LinkedIn is very popular at US and India, so we have two databases to reduce the latency. Person A is at US, and wrote a post, and it is immediately visible at his/her feed. Imagine Indian database syncs up with US database every 10 minutes. Or we have a queue. We can replicate the data to India, then the data is visible.

  • Semi-async replication: Transaction is considered successful after getting replicated on x number of nodes.

Pros

  • To scale out the database system: Replication allows multiple servers to handle the workload and provide access to the data, which can improve the performance and scalability of the system.
  • To improve availability: data can still be accessed even if one of the servers goes down or experiences an outage.
  • To improve reliability: Replication provides a backup copy of the data, which can be used to restore the database in the event of a failure or data loss.
  • Support disaster recovery: if the database is located in different location.

Cons

  • Overhead. It might add latency if we choose synchronous replication. It might result in inconsistency if we choose asynchronous replication.
  • Complexity to system
  • Selecting new master

Types of replicas

  • Write Replicas (Master/Leader Nodes): Write replicas are replicas that support 'writes' (create/update/delete). They may or may not support 'reads'.
  • Read Replicas (Slave/Follower Nodes): Read replicas are replicas that only support 'reads'.

How to do it?

  • Master-slave replication (ONE DB gets update): In this type of replication, one database server (the master) is designated as the primary source of data, and one or more other servers (the slaves) are used to replicate the data from the master. The slaves can be used to handle read-only queries, while updates are only made on the master. NOTE! We can also modify, so that leader election algorithm can be used to select slave as new master.
  • Peer-to-peer replication: In this type of replication, all servers in the system are equal peers and can be used to update and read data. No specially designed master. Changes made on one server are replicated to the other servers in the system.
  • Multi-master replication: Multiple database servers can accept write queries simultaneously, and propagate to other servers. This can be used to improve availability, but can also lead to conflicts if two servers make conflicting updates to the same data.

Sharding = multiple databases

In the context of databases, sharding refers to the practice of storing data across multiple servers, or "shards.”

Pros. Why?

  • To avoid the load on a database node, we can split the data in a way that the load is distributed across different nodes
  • Scale horizontal read and write performance.
  • Increase the amount of data that can be stored.

Cons. Difficulties.

  • Joins
  • Data should be distributed equally. We need sharding strategy. it requires careful planning to ensure that the data is distributed in a way that is both effective and efficient.
  • Reverse proxy is needed to distribute request from application server to database servers. It is a special software to manage the data distribution and communication between the different shards.

How to split the data? Sharding strategies

  1. Range-based sharding: This involves dividing the data into ranges based on a specific key, such as a timestamp or an ID number, and storing each range on a separate server.
  2. Hash-based sharding: This involves using a hash function to determine which server a piece of data should be stored on.
  3. Directory-based sharding: This involves maintaining a mapping of keys to servers in a separate "directory" server. When a request is made for a piece of data, the directory server is used to determine which server the data is stored on.
  4. Multi-shard key-based sharding: This involves storing multiple copies of each piece of data across multiple shards, with each copy being indexed by a different key.
  5. Zone sharding: This involves grouping data based on the geographic location of the user or the location of the data center where the data is stored. This can be useful for minimizing latency and ensuring that users are served by the nearest data center.
  6. Content-based sharding: This involves dividing data based on the type or category of the data, such as by user, product, or location. This can be useful for optimizing queries and indexing.

It's also worth noting that databases can be sharded in different ways for different purposes. For example, a database might use range-based sharding for writes and hash-based sharding for reads to optimize performance.

  • Consistent hashing based distribution problem is that if some shard goes down lose the data. Maybe we could replicate the shards.

Partitioning = one database

Partitioning = process of dividing a large table into smaller pieces, known as partitions, within 1 database server.

Why?

  • Improve query performance, allow for more efficient management of large tables
  • Enable parallel processing of data

Types

  1. Both sharding and partitioning can be either horizontal (meaning the data is divided into rows) or vertical (meaning the data is divided into columns). Horizontal sharding and partitioning involve dividing a table into smaller tables with fewer rows, while vertical sharding and partitioning involve dividing a table into smaller tables with fewer columns.
  2. Range-based partitioning: This involves dividing a table into partitions based on a range of values for a specific key, such as a date or an ID number.
  3. Hash-based partitioning: This involves using a hash function to determine which partition a piece of data belongs in.
SuperMade with Super