Database Sharding Explained: How Splitting a Database Helps It Scale

by Souy Soeng /October 13, 2026

Diagram of an app server sending queries through a shard router to three database shards on separate servers

Database sharding is a way of splitting one large database into smaller pieces, called shards, with each shard stored on its own server. Each shard holds only part of the data, so the storage, memory and traffic are spread across several machines instead of piling up on one.

This guide explains why sharding exists, how a shard key decides where each row lives, and how the three main strategies (range, hash and directory) work. It also covers where sharding is used and the mistakes that cause the most trouble, especially choosing the wrong shard key.

In short

A single database server has fixed limits on storage, memory and traffic.
Sharding spreads one database across several servers, and each shard holds different rows.
The shard key, such as user ID, decides which shard every row is saved on and read from.
Range, hash and directory sharding each have different strengths and trade-offs.
The most common problems are hotspots, cross-shard queries and sharding before you need to.

Why does a growing app slow its database down?

This problem usually appears when things are going well. More people sign up, and every new user brings more data and more requests. In a simple setup, all of that lands on one database. At some point pages load slowly, and when you look closer, the database is at its limit and can't answer requests fast enough.

Users feel this as a slow app, and they don't wait long. A slow checkout or a frozen feed can be enough to make someone close the app. If the database is pushed far enough past its limit, it can stop responding entirely, and then nothing works at all. If your data layer can't grow, your app can't grow either.

The root cause is physical. Every database runs on a server, and a server is a machine with a fixed ceiling. You can upgrade it for a while, but eventually there is no bigger machine to buy. Three limits matter most for a database, and sharding is designed to get around all of them at once.

Storage: every new user, order and message is saved, and eventually the disks fill up.
Memory: databases are fastest when busy data sits in RAM, and less of it fits as data grows.
Traffic: every read and write uses the same processor and network, which can be overwhelmed.

Scale up or scale out?

When a database reaches those limits, there are two basic options. You can make the one server bigger, which is called scaling up. Or you can add more servers and share the work between them, which is called scaling out. Both are valid, and most teams try scaling up first, because it is simpler.

Scaling up means moving to a more powerful server with more storage, more memory and faster processors. Your app doesn't need to change, because it just talks to a stronger database. The downside is that bigger machines get expensive, and there is always a largest one you can buy. You also still depend on a single machine, so if it fails, everything fails.

Scaling out means adding ordinary servers, each doing part of the work. You can keep adding servers as you grow, so there is no single hard ceiling. For a database, this raises a practical question: which server should hold which data? Sharding answers it by giving each server its own slice of the rows.

What is database sharding?

Sharding means splitting one large database into smaller pieces called shards, with each shard running on its own server. Every shard has the same tables, but each one holds different rows. A helpful picture is a library that has outgrown its building. Instead of one huge library, you open several branches, and each branch keeps a different part of the collection.

In a sharded system, the app server doesn't talk to the shards directly. It sends each query to a shard router. The router knows how the data was split, so it forwards each query to the right shard, for example Shard 1, Shard 2 or Shard 3.

Because each shard holds only part of the data, it is smaller, more of its busy data fits in memory, and it handles only part of the traffic. The work is shared, and every shard you add adds capacity. That is the main reason large apps use sharding when one server is no longer enough.

What is a shard key and why does it matter?

The router needs a rule for deciding where each row belongs. That rule is based on the shard key, a column you choose to decide where every piece of data lives. User ID is a common choice. Every row about a user carries that ID, so the router can read it and know exactly which shard to use.

The important detail is that one rule handles both saving and finding data. If the rule puts a user on Shard 2, that user's data is always written there and always read from there. Changing the shard key later means moving huge amounts of data, which is why choosing it is the biggest decision in sharding.

Here is a simple example. An app splits users by the first letter of their name: A to M on one shard, N to Z on another. When Maya opens her profile, the router checks the letter M and sends the request only to Shard A–M. Shard N–Z is never touched. If Noah logs in at the same time, his request goes to the other shard, so both run in parallel.

Only one shard does the work for each lookup.
Writes and reads follow the same rule.
A clear rule makes the system easier to understand and debug.

How does range-based sharding work?

The A to M example is range-based sharding. Each shard owns a continuous range of shard key values, such as letters, dates or ID numbers. Another example is an online store that places each order on a shard based on which part of the year it was made. The router reads the order's date and drops it into the matching range.

The main strength is range queries. If you ask for all orders from March, the router knows they all live on one shard, so it doesn't need to ask any others. That makes reports and lookups over a span of values fast and simple.

The trade-off is uneven load. With date ranges, every new order goes into the newest range while older shards sit mostly quiet. With name ranges, if many names start with the same few letters, one shard does most of the work. Ranges are easy to understand, but they can create hotspots.

How does hash-based sharding spread data evenly?

Hash-based sharding is designed to fix that uneven load. Instead of using the key's value directly, you run it through a hash function, which turns it into a number that looks random. The same input always gives the same output, but similar inputs give very different outputs. That is what spreads the data out.

A worked example makes it concrete. Start with user ID 1027. The hash function turns it into 58213. With four shards, you take 58213 modulo 4, which leaves a remainder of 1. So user 1027 is stored on Shard 1, and every future lookup for that user goes back to Shard 1.

The result is a very even spread, so no single shard gets crowded. The cost is that neighbouring values end up scattered, so a range query has to ask every shard. Adding a shard also changes the remainders, which means a lot of data has to move. Hashing gives you balance but gives up easy range queries.

Hash the key: 1027 becomes 58213.
Divide by the number of shards: 58213 % 4 = 1.
Store the row on Shard 1.

Directory-based sharding and how the three compare

Directory-based sharding replaces the formula with a lookup table that says which shard each key, or group of keys, lives on. It works like the directory board in an office lobby: you don't calculate which floor a company is on, you check the board. The router checks the table before every query.

The strength is full control. To move a customer, you copy their data and update one entry. If one large customer is overloading a shard, you can give them a shard of their own. The trade-off is that every query depends on the table. It adds an extra step, and if the table is slow or goes down, everything stops.

No strategy is best everywhere. Range splits by value ranges and keeps range scans fast, but can skew into hotspots. Hash gives the most even spread but makes range scans slow. Directory can be as even as you configure it and makes adding shards easiest, but the table can become a bottleneck. Start with your most common queries and pick the strategy that keeps them on a single shard.

Range: fast range queries, risk of hotspots.
Hash: very even spread, range queries touch every shard.
Directory: flexible placement, depends on the lookup table.

Where is database sharding used in real life?

Sharding is mostly found in apps with very large numbers of users, where data and traffic outgrew a single server long ago. Large social networks store enormous numbers of posts, messages and profiles. Sharding by user ID fits well because most of the data on a profile belongs to that one user.

Big online stores handle huge numbers of orders and shopping carts, especially during sales. Splitting by customer keeps each customer's history together on one shard, so their checkout stays quick. Multiplayer games often split players or game worlds across servers, and some online games even use the word shard for a separate copy of the game world.

The common thread is that most requests are about one user or one customer. When data splits cleanly along a key like that, most queries stay on a single shard, which is exactly the situation where sharding works best.

Hotspots and other sharding mistakes to avoid

Sharding only helps if the work is actually shared. A hotspot is one shard getting far more traffic than the others, and it usually comes from the shard key. If popular rows fall in the same range, or one key is extremely busy, most requests land on one shard while the rest sit idle. A sharded system is only as fast as its busiest shard.

To catch hotspots, monitor each shard separately. An average across all shards can look healthy while one is overloaded, so uneven numbers between shards are your early warning. To fix one, choose a key with many values that are used about equally, or use hashing to spread them out.

Three other mistakes are common and predictable. A poor shard key, with few distinct values or one your queries don't use, leads to hotspots and painful migrations. Cross-shard queries, like a report across all users, must ask every shard and combine results. And sharding too early adds real complexity before you need it.

Study your real queries and data before choosing a shard key.
Keep data that is used together on the same shard.
Try better indexes, caching or a bigger server before sharding.

Key takeaways

One database server has hard limits on storage, memory and traffic.
Sharding splits data across several servers so each shard holds different rows.
The shard key decides where data is saved and where it is read from.
Range, hash and directory sharding trade off speed, balance and flexibility.
Watch each shard separately to catch hotspots early.
Plan the shard key carefully, and shard only when you truly need to.

Frequently asked questions

What is the difference between sharding and scaling up?

Scaling up means moving your database to a bigger, more powerful server, and your app doesn't change. Sharding is a form of scaling out: it splits the data across several servers so each one handles part of the load. Scaling up eventually hits a ceiling, while sharding lets you keep adding servers.

How do I choose a good shard key?

Look at your real queries and data first. A good shard key has many distinct values that are used about equally, and it matches how your app usually looks up data, such as user ID. Avoid keys with only a few values, or keys your queries don't use.

Is hash-based sharding better than range-based sharding?

Neither is better in every case. Hash-based sharding spreads data very evenly, but range queries have to check every shard. Range-based sharding keeps range queries on one shard, but it can create hotspots when new or popular values cluster together.

What is a hotspot in a sharded database?

A hotspot is a shard that gets far more traffic than the others, usually because of how the shard key was chosen. That shard becomes overloaded while others sit idle, which slows the whole system down. Monitoring each shard separately helps you spot it early.

When should I start sharding my database?

Only when simpler options are no longer enough. Better indexes, caching or a bigger server are easier to manage and often solve the problem. Sharding adds real complexity to your code and operations, so it is best saved for when one server truly can't keep up.

Watch the full video on YouTube →

Tag

Ad