Learn how to architect, design, and build distributed systems for the real world.

Become a Distributed Systems Professional

In this module, we will examine ways to architect, design, and build software—particularly distributed systems.. This module is designed to prepare us for unpredictable users who might be a potential threat to our softwares. Our software will be under attack from the moment it is released. It needs to stand up to flash mobs or DDoS attacks by poorly secured IoT toaster ovens. We’ll examine  a software that failed the test and find ways to ensure our software survives contact with the real world.

Distributed Systems: Building Software for the Real World

# Shared resource effect

Another scaling effect that can jeopardize stability is the “shared resource” effect. Commonly seen in the guise of a service-oriented architecture or “common services” project, the shared resource is some facility that all members of a horizontally scalable layer need to use. With some application servers, the shared resource will be a cluster manager or a lock manager. When the shared resource gets overloaded, it’ll become a bottleneck limiting capacity. The following figure should give an idea of how the callers can put a hurting on the shared resource.

When the shared resource is redundant and nonexclusive, meaning it can service several of its consumers at once, then there’s no problem. If it saturates, we can add more and scale the bottleneck.

# Shared-nothing architecture

The most scalable architecture is the **shared-nothing architecture**. Each server operates independently, without need for coordination or calls to any centralized services. In a shared nothing architecture, capacity scales more or less linearly with the number of servers.
The trouble with a shared-nothing architecture is that it might scale better at the cost of failover. For example, consider session failover. A user’s session resides in memory on an application server. When that server goes down, the next request from the user will be directed to another server. Obviously, we’d like that transition to be invisible to the user, so the user’s session should be loaded into the new application server. That requires some kind of coordination between the original application server and some other device. Perhaps the application server sends the user’s session to a session backup server after each page request. Maybe it serializes the session into a database table or shares its sessions with another designated application server.

# Session failures

There are numerous strategies for session failover, but they all involve getting the user’s session off the original server. Most of the time, that implies some level of shared resources. We can approximate a shared-nothing architecture by reducing the fan-in of shared resources, for example, cutting down the number of servers calling on the shared resource. In the example of session failover, we could do this by designating pairs of application servers that each act as the failover server for the other. Too often, though, the shared resource will be allocated for exclusive use while a client is processing some unit of work. 

In these cases, the probability of contention scales with the number of transactions processed by the layer and the number of clients in that layer. When the shared resource saturates, we get a connection backlog. When the backlog exceeds the listen queue, we have failed transactions. At that point, nearly anything can happen. It depends on what function the caller needs the shared resource to provide. Particularly in the case of cache managers (providing coherency for distributed caches), failed transactions lead to stale data, or worse, loss of data integrity.

# Tips to remember
## Examine production vs. QA environments to spot Scaling Effects

Scaling Effects can cause trouble when we move from small one-to-one development and test environments to full-sized production environments. Patterns that work fine in small environments or one-to-one environments might slow down or fail completely when we move to production sizes.

## Watch out for point-to-point communication

Point-to-point communication scales badly, since the number of connections increases by a factor of the square of the number of participants. Consider how large our system can grow while still using point-to-point connections. It might be sufficient. Once we’re dealing with tens of servers, we will probably need to replace it with some kind of one-to-many communication.


Learn about shared resources, shared-nothing architecture, and session failures.

Shared Resources

Shared resource effect