Gain insights into distributed systems, learn architectural patterns like replication and sharding, and explore Apache Spark and Druid to design and develop effective distributed applications.

data_eng.tar.gz

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

AWS_DEFAULT_REGION

Apache_Druid

Mapreduce

aws_load_balancer

service_sharding

cqrs

apache_spark

apache_kafka

Gone are the days when monolithic applications were the norm. Most applications that we use today are too complex for that type of construction. Instead, most applications that appear as a single unit are actually built as a collection of inter-operating but independent computational components. Developers are expected to have mastery over a programming language or two. Moreover, they should also know the basics of distributed systems in order to succeed.

In this course, you’ll learn what a distributed system is, followed by the challenges unique to distributed systems. You’ll cover popular architectural patterns for achieving targets like replication and sharding. You’ll finish with a study of two popular distributed systems: Apache Spark and Apache Druid.

With the knowledge gained through this course, you’ll be able to design and develop distributed systems. You’ll be able to make optimal design and architectural choices based on the application requirements.

Introduction to Distributed Systems for Dummies

**(True or False)** If there is a partial failure, replication is useless in distributed systems.

(True or False) If there is a partial failure, replication is useless in distributed systems.

In single-leader replication, few write-requests can be sent to the followers.

In synchronous replication, followers fall behind sometimes.

Multi-leader replication is the simplest form of replication in terms of handling writes.

In leaderless replication, any replica can accept write-requests.

**(Select all that apply)** Which of the following statements are correct?

(Select all that apply) Which of the following statements are correct?

**(True or False)** In asynchronous replication, a follower might fall behind the leader.

(True or False) In asynchronous replication, a follower might fall behind the leader.

As soon as all the followers have processed the write-request.

As soon as the leader has processed the write.

As soon as one of the followers has processed the write-request.

As soon as a predefined number of followers have processed the request.

**(Select all that apply)** In hybrid replication, when is a write-request deemed as successful?

(Select all that apply) In hybrid replication, when is a write-request deemed as successful?

Elect a new node from the followers to be the leader.

Wait for the leader to come back and block all writes.

Promote the elected node to be the new leader of the system.

Route some of the requests to the new leader, and the rest to the old leader when it comes back.

Route all the requests to the new leader.

If the leader fails, what should the system do?

If the leader fails, what should the system do?

Evaluate your understanding of replication in distributed systems.

Introduction

What Distributed Systems Achieve for Us

Data in Distributed Systems

Communication Between Nodes

Data Processing in Large Scale

Distributed System Architectural Patterns

Case Study 1: Apache Spark

Case Study 2: Apache Druid

Conclusion

Quick Quiz #6: Replication