Why every developer should learn System Design

We live in a world where everything is dependent on devices that are always connected, around the clock. Our wristwatches, lighting systems, and smart refrigerators are connected to the cloud – and then sync to a mobile app on our phones. Just by reading this article, you’re relying on the Internet through a WiFi or cellular network. All of these services that support us each day are possible because of distributed systems.

A distributed system is a system with components located on various networked devices. When designed well, a distributed system is scalable and can adapt to support more users and functionalities, while remaining reliable and available at all hours.

System Design is the art of designing distributed systems.

Today, nearly every application is entangled with a distributed system. With that, System Design is becoming a non-negotiable skill set for proficient developers.

I have conducted System Design Interviews for hundreds of candidates between my time at Educative, Microsoft, and Facebook. Today, you can rarely advance into a senior developer role without passing a System Design Interview. But in reality, every developer, regardless of their level, should learn System Design. And they shouldn’t wait for the System Design Interview to learn it.

All of today’s development roles rely on distributed systems. So, no matter what type of developer you are, learning System Design will improve the quality of your work. So let’s talk about how you can gain an edge by learning System Design.

Why we need System Design#

I worked on distributed systems at Microsoft and Facebook for eight years, but my university program didn’t prepare me to work with distributed systems. I mostly learned about distributed systems and their design on the job and by reading white papers.

Before 2010 - when most of today’s software engineering managers were just starting, and when most of today’s leaders and instructors were managers - distributed systems were rarely discussed outside of Amazon, Google, and Microsoft. At that time, most applications were desktop applications that were not talking to the cloud. But as we know, technology changes quickly, and now virtually every application is becoming increasingly dependent on distributed system. As this trend continues into the future, we can expect even more applications to interact with distributed systems. With this in mind, a well-informed developer should know about distributed systems.

In the early days of distributed systems, engineers were navigating through the dark to build distributed systems from scratch.

As part of the pre-launch team for Microsoft Azure, I was lucky to be among this first generation of system engineers. We faced several challenges, failures, and unknowns in implementation in this uncharted territory. But because of the findings of this generation, we now have the discipline of System Design.

Through System Design, we can use design patterns and reusable components (or building blocks) to help us build massive scalable systems.

If you’re a committed developer, there’s no good reason to delay understanding the workings of the greater system to which you’re contributing.

In an evergreen quote by the late Danny Thorpe, the importance of understanding the greater system is clear:

“Programming without an overall architecture or design in mind is like exploring a cave with only a flashlight: You don’t know where you’ve been, you don’t know where you’re going, and you don’t know quite where you are.” – Danny Thorpe

System Design is the overall architecture for distributed systems. And not knowing System Design as a programmer in a distributed world is analogous to the difference between a line cook and head chef of a restaurant. As a head chef, you understand the greater system. You know how each and every staff member operates and relies on one another. You know each of their expected duties, tools, and methods. You know who comes in as backup when one person can’t fulfill a duty. And without this perspective, you’re a line cook. While you may know your particular tasks and methods well, you lack the vision of the big picture to which you’re contributing.

It’s clear why you need to know System Design if you want to advance into senior or leadership roles. But even if advancing into a higher role isn’t important to you, you should want to know about System Design, because you’re a key player within its architecture.

Like most leaders, the main quality I look for in developers is a drive to learn and know about the world to which they’re contributing.

If you want to be a great developer, you need to have a drive to fill your knowledge gaps. A great developer is curious to a fault.

If you’re a front-end developer, your work never happens in isolation. Even when you’re rendering a web page, you’re interacting with several different distributed systems through APIs.