Gain insights into distributed systems, learn architectural patterns like replication and sharding, and explore Apache Spark and Druid to design and develop effective distributed applications.

data_eng.tar.gz

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

AWS_DEFAULT_REGION

Apache_Druid

Mapreduce

aws_load_balancer

service_sharding

cqrs

apache_spark

apache_kafka

Gone are the days when monolithic applications were the norm. Most applications that we use today are too complex for that type of construction. Instead, most applications that appear as a single unit are actually built as a collection of inter-operating but independent computational components. Developers are expected to have mastery over a programming language or two. Moreover, they should also know the basics of distributed systems in order to succeed.

In this course, you’ll learn what a distributed system is, followed by the challenges unique to distributed systems. You’ll cover popular architectural patterns for achieving targets like replication and sharding. You’ll finish with a study of two popular distributed systems: Apache Spark and Apache Druid.

With the knowledge gained through this course, you’ll be able to design and develop distributed systems. You’ll be able to make optimal design and architectural choices based on the application requirements.

Introduction to Distributed Systems for Dummies

**(True or False)** In distributed systems, things rarely go wrong.

(True or False) In distributed systems, things rarely go wrong.

A node in a distributed system can handle an infinite number of user requests.

Faults are common in distributed systems.

In a single-node architecture, the system goes down a lot easier.

**(Select all that apply)** Which of the following is false?

(Select all that apply) Which of the following is false?

**(True or False)** In a distributed system, fault tolerance is good to have, not a must.

(True or False) In a distributed system, fault tolerance is good to have, not a must.

**(True or False)** As a distributed system owner, when things go wrong in the system, you should quickly halt everything and go on a repair mode.

(True or False) As a distributed system owner, when things go wrong in the system, you should quickly halt everything and go on a repair mode.

Let it continue the way it is. At least the system has more uptime.

Make the system capable of detecting data corruption.

Stop the part of the system that contains data corruption immediately without any action and let the other parts continue to operate.

Create some noise. Trigger alerts and send logs so that system owners can take action.

**(Select all that apply)** You build a system that continues to operate silently even if there is data corruption. What would you change in the system to make it more reliable?

(Select all that apply) You build a system that continues to operate silently even if there is data corruption. What would you change in the system to make it more reliable?

In large distributed systems, hard drives never die.

Having data replication helps reliability.

It's not important to have unit tests in your distributed system code.

Integration tests simulate user behavior in a distributed system.

**(Select all that apply)** Select the false statements.

(Select all that apply) Select the false statements.

Check your understanding of fault tolerance and reliability.

Introduction

What Distributed Systems Achieve for Us

Data in Distributed Systems

Communication Between Nodes

Data Processing in Large Scale

Distributed System Architectural Patterns

Case Study 1: Apache Spark

Case Study 2: Apache Druid

Conclusion

Quick Quiz #2: Fault Tolerance and Reliability