Grokking the Product Architecture Design Interview/

...

/

File API Design Decisions

File API Design Decisions

Learn what design decisions will best suit the file API.

We'll cover the following...

In this lesson, we examine the overall operation of the file API and the interaction of the individual components. This will help us make certain design decisions while diving deeper into the design challenges.

Design overview

There are many architectures that lead to good API design, but we can’t simply pick one and start developing. API design is often influenced by business requirements, technologies used, etc. Therefore, defining a simple step-by-step guide for choosing an architecture may not be possible. Often, a combination of multiple architectures work in tandem to achieve organizational goals. Let’s go over the design considerations of a file API and determine the suitable ingredients (architecture style, data format, etc.) to satisfy our needs.

The following diagram contains the detailed workflow of the components and services involved in the file API design.

Press + to interact
Detailed workflow of a file API
Detailed workflow of a file API

Let’s look at the functionality of these components and services in the table below.

Components and Services Details

Component or Service

Details


File servers

  • Accept and process file API requests forwarded by the API gateway
  • Split requests into metadata (user and file information) and file content

Processing server

  • Performs encoding/decoding of data into different encoding schemes
  • Encrypt/decrypt data at-rest to prevent unauthorized access

User to file mapping system (UFMS)

  • Maps users to files and where those files can be found in storage

Blob storage

  • Stores binary data objects (file content)

Temporary storage

  • Stores files and objects temporarily before processing

User server

  • Handles user related requests forwarded by the API gateway

SQL database

  • Stores information related to files and users (metadata)

CDN servers

  • Geographically distributed network for serving data to the user from a nearby location instead of serving it from the blob storage


API gateway

  • Forwards client requests to the appropriate application server
  • Authenticates and authorizes client requests
  • Rate-limits client requests to keep the server from overburdening

Client

  • The consumer of the API service
  • Can perform lossless compression before sending a file

Workflow

For a better understanding, we can divide the workflow shown in the illustration above into the following two parts.

  • Client to API gateway: Before performing any operation (upload, download, and so on), the API needs to authenticate the client and ensure that it has read/write access to the data. After successful verification, a user can upload, download, list, or delete a file from the server through HTTP requests.

  • API gateway to downstream services: As the request reaches the API gateway, the gateway forwards the request to the appropriate API services. If the request is to upload a file, the application server extracts the metadata (such as user ID, file name, file size, file location, ...