Creating API with Node.js: A System Design approach

As a software engineer, optimizing your Node-API to handle high volumes of requests per second is crucial for developing high-performance applications that can effectively manage growing traffic loads. In this blog post, we'll explore effective strategies and techniques to help you scale your Node-API.

My journey with API development began in the late 90s with ISAPI (Internet Server Application Programming Interface), a technology that allowed me to create custom web server extensions and filters. Although working with ISAPI wasn't the most exciting experience, it sparked my interest in building APIs and gave me a sense of control over my own web server. Fast forward to today, and I'm excited to see how far API development has come, particularly with the rise of Node.js. The flexibility and scalability of Node.js have made it an ideal choice for building high-performance APIs, and I'm eager to share my knowledge on how to do Node.js performance optimization.

In this blog post, we'll delve into the techniques and best practices for scaling your Node-API, from optimizing server configuration to leveraging caching and load balancing. Whether you're a seasoned developer or just starting out, this guide will help you build high-performance applications that can handle growing traffic loads with ease.

Let’s begin by revisiting our understanding of scalability, exploring its crucial role in robust system design and how it impacts system architecture.

Scalability and System Design#

When we talk about scalability, we mean an application’s capability to handle a greater load, such as increased users, data, or requests. Scalability is achieved by creating systems that can expand seamlessly while maintaining optimal performance. Here are a few significant factors to consider:

Horizontal vs. vertical scaling#

Scaling can be of two types.

Horizontal scaling involves adding more machines (nodes) to distribute the load. Load balancers route requests to different instances, ensuring better resource utilization and fault tolerance.

Vertical scaling instead focuses on enhancing the capacity of a single machine (e.g., upgrading CPU, RAM). While vertical scaling has limits, it’s simpler to implement initially.

Database scaling#

Another aspect to consider regarding scaling is database scaling. Typically when we talk about this topic, we come across two types of database scaling: sharding and replication.

Sharding involves splitting data across multiple databases or shards. Each shard handles a subset of data, improving read/write performance.

Replication, on the other hand, involving creating copies (replicas) of the database to distribute read traffic. Master-slave replication ensures data consistency.

Caching#

Next, we can also use in-memory caches for scaling. Caching in Node.js allows us to temporarily store frequently used data rather than repeatedly computing it. Doing so lets us quickly retrieve the cached data without rerunning computations or querying the database. This significantly improves application performance. In-memory caching is a common approach, where data is stored in memory for rapid access. It’s particularly useful for scenarios where data rarely changes, such as lists of products, country calling codes, or store locations. In-memory databases offer benefits like faster reads/writes, lower latency, and scalability, but they are more volatile than traditional disk-based databases. To implement caching in Node.js, we can use packages like Redis or node-cache.

Connection pooling#

Connection pooling is a technique for managing database connections efficiently. It maintains a pool of reusable connections rather than creating and destroying connections for each database request. This approach significantly improves the performance and scalability of applications that interact with databases, as establishing a new connection can be resource-intensive and time-consuming.

How connection pooling works#

When an application requires database interaction, it asks for a connection from the pool. If a connection can be used, it is returned to the application. Instead of closing the connection, the application returns it to the pool once it’s done. Allowing the connection to be reused reduces the overhead of establishing new connections for future requests.

Benefits of connection pooling#

Performance improvement: Connection pooling reduces latency by reusing connections, resulting in faster response times for database queries.

Resource management: Connection pooling aids in efficient management of database resources by restricting active connections, preventing server overwhelm, and optimizing performance.

Scalability: Managing connections dynamically based on demand, connection pooling allows applications to scale more efficiently as concurrent users or requests increase.

Reduced overheads: Connection pooling, which minimizes connection creation and destruction, reduces overall resource consumption on the application and database server.

In short, connection pooling is a critical technique for optimizing database interactions in applications, boosting performance, resource management, and scalability. It is frequently employed in web applications, microservices, and any system that demands regular database access.

Optimizing system performance#

System performance can be optimized in several ways, but three topics require special discussion in this domain.

Asynchronous messaging#

Asynchronous messaging enables the exchange of information between system components with no simultaneous engagement. In this model, messages can be sent and received independently, allowing senders to carry on with other tasks without waiting for a response from the receiver. Decoupling components makes the system more scalable and resilient by distributing loads efficiently and improving fault tolerance. Message queues or event-driven architectures are often used to implement asynchronous messaging, allowing messages to be temporarily stored until the receiving component is prepared to handle them. This method is especially helpful in distributed systems, microservices architectures, and applications with high throughput and responsiveness needs, as it aids in workload management and enhances overall performance.

Concurrency#

Concurrency refers to a system’s capability to manage multiple tasks or processes simultaneously, enabling them to advance independently without disruption. In computing, concurrency is crucial in optimizing resource usage and enhancing application responsiveness, especially in environments involving multiple threads or distributed systems. By enabling systems to multitask, such as processing user requests, managing data transactions, and executing background tasks, it can significantly enhance performance and user experience. Dealing with concurrency also presents challenges like race conditions, deadlocks, and the requirement for synchronization mechanisms to maintain data integrity and consistency. The ability to effectively manage concurrency is vital when developing applications that need to scale and perform well under growing demands.

Responsiveness#

Responsiveness pertains to how fast a system or application can respond to user inputs or events, ensuring timely feedback and a smooth user experience. In the realm of software development, it is crucial to prioritize responsiveness as it directly affects user engagement and perceived control. Delays or unresponsive interfaces can cause frustration and reduced satisfaction levels. To achieve high responsiveness, optimizing performance by utilizing techniques like asynchronous processing, effective resource management, and reducing latency in data retrieval and rendering is important. In web applications, responsiveness can be improved by utilizing strategies such as lazy loading, client-side caching, and responsive design. These approaches guarantee the swift delivery of content across a range of devices. In the end, a highly responsive application enhances the user’s experience and increases user retention and engagement.

Content delivery networks (CDNs)#

A content delivery network (CDN) is a network of servers that stores content near users for faster access. CDNs expedite the delivery of assets required for loading Internet content, such as HTML pages, JavaScript files, style sheets, images, and videos, thereby boosting website performance. They play a vital role in serving the web traffic of major sites like Facebook, Netflix, and Amazon. CDNs come with benefits that include cost savings on bandwidth, increased availability of content, and enhanced protection against common malicious attacks, such as DDoS attacks. A CDN doesn’t replace the need for proper web hosting, but it does significantly improve website performance by delivering content closer to users and efficiently managing traffic.

System Design#

System Design is the process of creating a blueprint for developing complex software applications. It covers decisions related to architecture, components, data flow, and interactions. Effective System Design involves considering scalability, reliability, security, and performance. Engineers often create robust systems using domain-driven design, microservices, and database schema design.

Some of its key principles include:

Modularity#

Modularity means dividing the system into smaller, independent components (modules). Each module has a specific responsibility and communicates via well-defined interfaces.

Decoupling#

We can minimize dependencies between modules. Loose coupling allows for easier maintenance, testing, and scalability.

Microservices#

Build applications as a collection of small, loosely coupled services. Microservices enable independent scaling and deployment.

API design#

We can create clean, RESTful APIs with clear endpoints. When constructing an API, it’s necessary to make planning and architectural decisions for a better end result. The design of our basic API directly affects developer consumption and utilization. API design, like a website or product design, is crucial to user experience. Understanding the intended use case is crucial in developing a successful API. It helps to define the API’s contract by utilizing a specification such as OpenAPI, validate assumptions through mocks and tests, and provide clear documentation for each resource, method, parameter, and path.

If you would like to know more about system design, check the following resource.

What is Node.js?#

Node.js is a cross-platform JavaScript runtime environment that executes JavaScript code outside of a web browser and is open-source. Developers have the ability to build servers, web applications, command-line tools, and scripts. While traditional JavaScript is mainly confined to browser usage, Node.js empowers us to leverage JavaScript in multiple contexts, such as the front end, middleware, and back end. It’s a component of well-known web development stacks like MERN, MEVN, and MEAN stacks. The key features include the Google Chrome V8 JavaScript engine, npm (a package manager), and an event-driven, single-threaded I/O model.

What is an API?#

APIs (application programming interfaces) serve as bridges between different software systems, allowing them to communicate and interact. An API defines a set of rules and protocols that enable applications to request and exchange data or perform specific actions. These interfaces can be categorized into several types:

• Web APIs (RESTful APIs): These are commonly used for web applications. They follow the principles of Representational State Transfer (REST) and use HTTP methods (such as GET, POST, PUT, DELETE) to perform operations on resources (e.g., retrieving data from a server, updating records).

• Library APIs: These APIs provide pre-built functions and methods within programming libraries or frameworks. Developers use them to access specific features or functionalities (e.g., JavaScript APIs for manipulating the DOM, Python APIs for working with databases).

• Operating system APIs: OS-specific APIs allow applications to interact with the underlying operating system. Examples include Windows API (for Windows OS) and POSIX API (for UNIX-like systems).

• Hardware APIs: These enable communication with hardware components (e.g., graphics cards, sensors, printers). Device drivers often expose hardware APIs.

Third-party APIs: Developed by external providers, these APIs allow access to services like social media platforms (e.g., Twitter API, Facebook Graph API), payment gateways (e.g., PayPal API), or weather data (e.g., OpenWeatherMap API).

API scaling techniques#

With scalable APIs, applications can effectively manage higher traffic and data processing needs. Being adaptable is important for handling peak usage and ensuring consistent performance. Key guidelines for developing a scalable Node.js API include:

• Statelessness: All the information necessary to process a request is included in every client request in a stateless API. The server’s design is simplified by not requiring memory of past interactions, enabling requests to be processed by any available computing resource.

• Loose coupling: The concept of loose coupling involves minimizing dependencies when designing API components. Its flexibility allows for modifications or scaling of system parts without greatly impacting other components.

• Vertical and horizontal scaling: Both vertical scaling (expanding resources within the current infrastructure) and horizontal scaling (adding additional machines or nodes) are critical. To handle large-scale applications, it is essential to have the virtually limitless growth offered by horizontal scaling.

• Resource-oriented design: It involves organizing our API around resources instead of actions.

API scaling in Node.js#

Scaling a Node API involves ensuring that our application can handle growing traffic loads without compromising performance. Here are some key principles to consider:

A solid foundation starts with well-structured and efficient code and architecture. When our codebase is optimized, it becomes easier to scale our application without sacrificing performance. Breaking down complexity by dividing our application into smaller, self-contained modules is called modularity. Each module should have a specific responsibility and provide a well-defined interface for interaction with other modules. This modular approach allows independent development, testing, and scaling of individual components.

Decoupling reduces dependencies between modules. Loose coupling enables us to modify, replace, or scale individual modules without impacting the entire system. Node.js excels at handling asynchronous operations. By maximizing concurrency and responsiveness, we can optimize performance and scalability. Utilize non-blocking I/O capabilities to handle multiple requests efficiently.

Horizontal scaling involves distributing the load across multiple instances of our API. Use load balancers to route requests to different servers, ensuring better resource utilization and fault tolerance. Caching and database optimization involve implementing caching mechanisms (e.g., Redis, as discussed above) to reduce database queries. Optimize database queries and indexes to improve overall performance.

Choosing the right architecture#

One of the things that differs between different API designs is their architecture. Let’s compare REST, GraphQL, and other architectural patterns for API design.

REST#

In the context of web services, REST (Representational State Transfer) is an architectural style that emphasizes a stateless client-server interaction. It relies on standard HTTP methods (such as GET, POST, PUT, DELETE) to perform operations on resources (usually represented as URLs). RESTful APIs allow clients to access and manipulate data via well-defined endpoints, promoting scalability, simplicity, and loose coupling between components.

Key principles of REST:#

• Predefined endpoints: REST uses specific URLs to interact with resources, making it easy to understand and use. Each endpoint corresponds to a specific resource, such as /users or /posts.

• Standard HTTP verbs: REST relies on standard HTTP verbs (GET, POST, PUT, DELETE) to perform CRUD (Create, Read, Update, Delete) operations. This makes it easy to understand and use, as the verbs are widely recognized and used.

• Statelessness: Each request from the client contains all necessary information, as the server doesn't maintain session state. This makes it easy to scale and maintain, as the server doesn't need to keep track of client state.

• Resources are the core: In REST, resources are the core building blocks, and each has a unique URI. This makes it easy to identify and interact with specific resources.

• Unique URIs: Each resource has a unique URI, making it easy to identify and access specific resources.

• Robust Caching Capabilities: REST offers robust caching capabilities, making it easy to retrieve data efficiently. This reduces the load on the server and improves performance.

To summarize, the pros of using REST include proven design principles, ease of implementation, and widespread adoption. The cons include fixed endpoints, data over- or under-fetching, and versioning challenges.

GraphQL#

GraphQL uses a single flexible endpoint (/graphql) to respond to queries as a communication model. Clients specify the exact data they need. Clients define their data requirements using a query language. Responses match the structure of the query. It allows for a reduction in over-fetching by fetching only the requested data. GraphQL offers a dynamic Schema that evolves with the application and has no fixed endpoints.

In terms of its benefits, it is flexible, efficient, and allows precise data retrieval, making it Ideal for complex scenarios. However, it is not without cons: It requires more tooling and security considerations and can lead to complex queries.

Other patterns#

The above two are not the only possible architectures for API design. There are other patterns such as:

• gRPC is a high-performance, language-agnostic RPC (Remote Procedure Call) framework and uses Protocol Buffers for serialization.

• WebSocket APIs offers real-time communication over a single connection and is ideal for chat applications, live updates, and gaming.

• JSON-RPC is a lightweight protocol for remote procedure calls using JSON. Last but not least is SOAP (Simple Object Access Protocol), one of the older, XML-based protocols for exchanging structured information platform-independently.

Choosing the right architecture#

REST is reliable, efficient, and suitable for most use cases whereas GraphQL is Optimal for scenarios with complex data retrieval needs or evolving schemas. We can also opt for other Patterns considering them for specific requirements and constraints.

Techniques for scaling horizontally with Node.js#

Let’s examine some techniques for distributing load across multiple instances of our Node.js app.

Node.js clustering:#

The first of these techniques is clustering. To utilize all available CPU cores efficiently, Node.js introduced the cluster module. It allows deploying multiple instances of Node.js processes. The cluster module incorporates a load balancer that evenly distributes incoming requests among these instances running on different cores.

Load balancers#

Each system should have a load balancer to distribute traffic across the node instances. For distributed systems, there should be a master load balancer that distributes traffic across multiple node instances on different servers. The master balancer should always be running or have a dependable restart mechanism to keep the app stable.

Shared state management#

When multiple processes serve requests, we can no longer use memory to store state. Use an additional service like Redis (a popular choice) to store shared states, as it provides efficient in-memory data storage and supports features like queues, hashes, and pub/sub.

Database optimization strategies#

Designing efficient database schemas and optimizing queries.
 Efficient database schema design and query optimization are critical for building performant Node.js applications. Let’s explore some possible strategies:

Schema design#

Normalization involves breaking down data into smaller tables to minimize redundancy and improve data integrity, whereas denormalization helps reduce joins and improve query performance for read-heavy workloads.

Partitioning involves dividing large tables into smaller partitions based on specific criteria (e.g., date ranges) to enhance scalability. We have discussed sharding earlier. Let us develop a deeper understanding by going a bit deeper.

Partitioning vs. sharding: A deep dive#

The primary concept to grasp here is that database partitioning is a technique employed to break down a large database into smaller, more manageable segments. This approach enhances performance, manageability, and availability by enabling queries to work on smaller data subsets instead of the entire database. An added benefit of partitioning is its improved scalability, allowing for better resource utilization and load distribution among multiple servers.

Horizontal partitioning (Sharding)#

Horizontal partitioning, commonly known as sharding, is the practice of dividing a database table into smaller, more manageable parts based on rows. A shard comprises a specific set of data, with each shard typically being categorized based on a specific key, such as user ID or geographic location. To provide an example, an approach to structuring a user database involves partitioning it, separating users from various regions into different shards. This method enhances performance by targeting queries at the specific shard instead of scanning the entire dataset. Sharding can improve scalability by distributing the workload across multiple servers. This enables the system to handle a greater number of users simultaneously and manage larger datasets.

Vertical partitioning#

Conversely, vertical partitioning entails dividing a database table into smaller portions based on its columns. The approach involves dividing a table into several tables, each with a subset of the original table’s columns. As an example, a user profile table could be split into two tables: one for basic user information (such as name and email) and another for additional details (like preferences and settings). By implementing vertical partitioning, queries can be optimized to access only the required columns, resulting in reduced data processing and improved performance. Isolating sensitive information or frequently accessed data can also improve security and manageability.

In summary, both horizontal (sharding) and vertical partitioning are effective strategies for optimizing database performance and scalability. Each addresses different aspects of data management and access patterns.

Indexing helps create indexes on frequently queried columns to speed up data retrieval. We can also use composite indexes for multiple columns to optimize complex queries. Schemas need to be regularly reviewed and queries should be refactored for efficiency. Besides, it makes sense to consider caching frequently accessed data.

ORMs (object-relational mappers)#

Libraries like Sequelize provide an intuitive way to define models and perform queries. Sequelize supports options like sorting, pagination, and filtering, allowing customization of queries. However, consider using raw SQL queries for complex scenarios where performance and flexibility matter.

Database-level optimization#

Working at the database level is one way to improve scalability. We can adjust database configurations, such as connection pool size and query cache, to achieve optimal performance. Moreover, we can enhance Sequelize models to match query patterns, track query execution plans, and identify areas for further optimization.

Conclusion#

We have seen that scalability in System Design refers to a system’s ability to handle increasing workloads, user traffic, and data without compromising performance or stability. In the context of a Node.js API, achieving scalability involves considering several factors. We need to make the right decision for each step to achieve an optimal and scalable API design.

Explore our courses on designing high-performance APIs and more given below. I recommend the course Building Full-Stack Web Applications With Node.js and React to help you develop and deploy full-stack applications in production environments. If you're more focused on system design, the course Scalability & System Design for Developers will give you insights into how to build systems at scale.