Design of YouTube

High-level design

The high-level design shows how we will interconnect various components we identified in the previous lesson. We have started developing a solution to support the functional and non-functional requirements with this design.

The workflow for the abstract design is provided below:

1. The user uploads a video to the server.
2. The server stores the metadata and the accompanying user data to the database and, at the same time, hands over the video to the encoder for encoding (see 2.1 and 2.2 in the illustration above).
3. The encoder along with the transcoder, compresses the video and transforms it into multiple qualities (like 2160p, 1440p, 1080p, etc.). The videos are stored on blob storage (similar to GFS or S3).
4. Some popular videos may be forwarded to Content Distribution Network (CDN) which acts as a cache.
5. The CDN, because of its vicinity to the user, lets the user stream the video with low latency. However, CDN is not the only infrastructure for serving videos to the end-user as we will see in the detailed design.

API design

Let’s understand the design of APIs in terms of the functionalities we are providing. We will design APIs to translate our feature set into technical specifications. In this case, the REST APIs can be used for simplicity and speed purposes. Our API design section will help us understand how the client will request services from the backend application of YouTube. Let’s develop APIs for each of the following features:

• Upload video
• Stream video
• Search video
• View thumbnails
• Like/dislike video
• Comment video

Upload video

The POST method can upload a video to the /uploadVideo API:

uploadVideo(user_id, video_file, category_id, title, description, tags, default_language, privacy_settings)

We would like to explain the following parameters here:

• user_id is the user that is uploading the video.
• video_file is the video file that the user wants to upload.

The video file is broken down into smaller packets and uploaded to the server in order. In case of failure, YouTube can store data for a limited time and resume the upload if the user retries. To understand the concept in detail, you may want to read about Asynchronous APIsThe kind of APIs where the server cannot fulfill the request immediately either because the data is too large or isn’t readily available. In that case, the client gets an immediate response that the server has accepted the request while the server responds with the data later on. Such a mechanism saves CPU and network bandwidth and does not keep the client waiting. ...