Learn how to architect, design, and build distributed systems for the real world.

Become a Distributed Systems Professional

In this module, we will examine ways to architect, design, and build software—particularly distributed systems.. This module is designed to prepare us for unpredictable users who might be a potential threat to our softwares. Our software will be under attack from the moment it is released. It needs to stand up to flash mobs or DDoS attacks by poorly secured IoT toaster ovens. We’ll examine  a software that failed the test and find ways to ensure our software survives contact with the real world.

Distributed Systems: Building Software for the Real World

# Local visibility
Transparency arises from deliberate design and architecture. Adding transparency late in development is about as effective as adding quality. Maybe it can be done, but only with greater effort and cost than if it’d been built in from the beginning.

Visibility inside one application or server is not enough. Strictly local visibility leads to strictly local optimization. For example, a retailer ran a major project to get items appearing on the site faster. The nightly update was running until 5 or 6 a.m., when it needed to complete closer to midnight. This project optimized the string of batch jobs that fed content to the site. The project met its goals, in that the batch jobs finished two hours earlier. Items still did not appear on the site, however, until a long-running parallel process finished, at 5 or 6 a.m. The local optimization on the batch jobs had no global effect.
# Scaling effects
Visibility into one application at a time can also mask problems with scaling effects. For instance, observing cache flushes on one application server would not reveal that each server was knocking items out of all the other servers’ caches. Every time an item was displayed, it was accidentally being updated, therefore causing a cache invalidation notice to all other servers. As soon as all the caches’ statistics appeared on one page, the problem was obvious. Without that visibility, we would’ve added many servers to reach the necessary capacity, and each server would’ve made the problem worse.


Learn about visibility in systems and optimization, scaling effects, and coupling.


Designing for Transparency

Learn about visibility in systems and optimization, scaling effects, and coupling.