Skip to content
System Design

Database Sharding

One-Liner

A technique for horizontally partitioning a database to distribute load.

What It Is

The process of breaking up a large database into smaller, faster, more easily managed parts called shards. Each shard is an independent database, and together they make up a single logical database.

Why It Exists

To allow a database to scale horizontally. A single database server can only be scaled up (vertically) so much. Sharding allows a database to scale out across many servers.

How It Works

  • A “shard key” is chosen from the data (e.g., user_id).
  • A function is used to map each shard key to a specific shard.
  • The application or a routing layer is responsible for directing queries to the correct shard.

Tradeoffs

Pros

  • Enables massive horizontal scaling of reads and writes.

Cons

  • Significantly increases application complexity.
  • Makes cross-shard joins difficult or impossible.
  • Can lead to “hot spots” if the shard key is not chosen well.

Failure Modes

  • Hot Spots: An uneven distribution of data or traffic causes one shard to be overloaded while others are idle.
  • Cross-shard Join Complexity: Queries that need to join data across multiple shards are very expensive and complex.
  • Resharding Complexity: Adding or removing shards can be a very complex and risky operation.

Interview Traps

  • Suggesting sharding as the first solution for scaling a database (read replicas are usually a better first step).
  • Not discussing the importance of choosing a good shard key.

Real-World Usage

  • Large-scale applications with massive datasets, such as social networks (e.g., Facebook’s user database).

Anti-Patterns

  • Sharding a small database that could be scaled vertically.
  • Choosing a shard key that will lead to hot spots (e.g., sharding by country for a business that is mostly in one country).
  • Read Replicas
  • Consistent Hashing
  • SQL vs. NoSQL