Iterate and Scale

It is the most critical phase of system design. Sometimes, the interviewer wants us to focus only on this part.

We have created a high-level design by now, but it may not be scalable and reliable. We are supposed to dip deep into our high-level design and improve, but it is nearly impossible to go into each component in the interview. Sometimes, the interviewer will suggest a particular component they’d like us to expand on. If not, we should expand on the database part.

Scalability

It depends on the context on what and where you are trying to scale, but we will cover topics based on the experience we need in this phase.

Partitioning

Partitioning will be used when we have to scale our database and reduce the response time for clients.

Traditionally, if we store all of the data into one database, it will take a substantial amount of time to look up and query the data. Most people distribute their data among multiple tables to improve responsiveness and for better manageability.

Partitioning divides the big table into multiple parts, but there are many ways to divide it.

Horizontal partitioning

Horizontal partitioning is also known as Sharding. In this technique, all tables will have the same set of columns. The schema for different tables is the same, but they are divided based on the range.

If the range is not carefully decided, it may lead to unbalanced tables.

Vertical partitioning

In this partitioning, we divide the tables based on the columns. It means different tables will have distinct columns. We can divide our table columns based on usability.

Also, we can keep the most frequent columns on faster storage like SSD and other ones in the database or the cost-effective storage.

Directory-based partitioning

Directory-based partitioning is a technique that is loose and can be changed as per your needs. It solves the problem of the above types of prioritizing. In this, you have a directory that contains the mapping and works as a lookup service that is aware of the partitioning.