Requirements of a Blob Store's Design

Requirements

Let’s understand the functional and non-functional requirements below:

Functional requirements

Here are the functional requirements of the design of a blob store:

• Create a containerA container is like a folder in a file system used to group blobs. Don’t mix up this container with a Docker container.: The users should be able to create containers in order to group blobs. For example, if an application wants to store user-specific data, it should be able to store blobs for different user accounts in different containers. Additionally, a user may want to group video blobs and separate them from a group of image blobs. A single blob store user can create many containers, and each container can have many blobs, as shown in the following illustration. For the sake of simplicity, we assume that we can’t create a container inside a container.

• Put data: The blob store should allow users to upload blobs to the created containers.
• Get data: The system should generate a URL for the uploaded blob, so that the user can access that blob later through this URL.
• Delete data: The users should be able to delete a blob. If the user wants to keep the data for a specified period of time (retention time), our system should support this functionality.
• List blobs: The user should be able to get a list of blobs inside a specific container.
• Delete a container: The users should be able to delete a container and all the blobs inside it.
• List containers: The system should allow the users to list all the containers under a specific account.

Non-functional requirements

Here are the non-functional requirements of a blob store system:

• Availability: Our system should be highly available.
• Durability: The data, once uploaded, shouldn’t be lost unless users explicitly delete that data.
• Scalability: The system should be capable of handling billions of blobs.
• Throughput: For transferring gigabytes of data, we should ensure a high data throughput.
• Reliability: Since failures are a norm in distributed systems, our design should detect and recover from failures promptly.
• Consistency: The system should be strongly consistent. Different users should see the same view of a blob.

Resource estimation

Let’s estimate the total number of servers, storage, and bandwidth required by a blob storage system. Because blobs can have all sorts of data, mentioning all of those types of data in our estimation may not be practical. Therefore, we’ll use YouTube as an example, which stores videos and thumbnails on the blob store. Furthermore, we’ll make the following assumptions to complete our estimations.

Assumptions:

• The number of daily active users who upload or watch videos is five million.
• The number of requests per second that a single blob store server can handle is 500This number can be significantly higher, depending upon the blob store. For example, Microsoft Azure can handle a maximum of 20,000 IOPS..
• The average size of a video is 50 MB.
• The average size of a thumbnail is 20 KB.
• The number of videos uploaded per day is 250,000.
• The number of read requests by a single user per day is 20.