Gain insights into distributed systems, learn architectural patterns like replication and sharding, and explore Apache Spark and Druid to design and develop effective distributed applications.

data_eng.tar.gz

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

AWS_DEFAULT_REGION

Apache_Druid

Mapreduce

aws_load_balancer

service_sharding

cqrs

apache_spark

apache_kafka

Gone are the days when monolithic applications were the norm. Most applications that we use today are too complex for that type of construction. Instead, most applications that appear as a single unit are actually built as a collection of inter-operating but independent computational components. Developers are expected to have mastery over a programming language or two. Moreover, they should also know the basics of distributed systems in order to succeed.

In this course, you’ll learn what a distributed system is, followed by the challenges unique to distributed systems. You’ll cover popular architectural patterns for achieving targets like replication and sharding. You’ll finish with a study of two popular distributed systems: Apache Spark and Apache Druid.

With the knowledge gained through this course, you’ll be able to design and develop distributed systems. You’ll be able to make optimal design and architectural choices based on the application requirements.

Introduction to Distributed Systems for Dummies

# A warm welcome

Hello there, welcome to the course!


If you are new to the world of distributed systems, this course is definitely for you.

On the other hand, this course is also for you if you need to brush up your core understanding of distributed systems and possibly learn some new stuff along the way.

This course assumes that you have no prior knowledge of distributed systems. 

You are probably a college student or an entry-level engineer in the tech industry, or perhaps you are new to large-scale system engineering altogether. 

In this course, we discuss the core concepts gradually from the ground up. After finishing the course, we expect that you will build a good grasp of the concepts of distributed systems and hopefully apply the knowledge in real life.

# A brief overview of the course

This course will make you proficient in distributed systems and help you decide whether your system should adopt a distributed architecture or not. Keeping this goal in mind, we have designed this course to carefully teach the core concepts of distributed systems:

- The first chapter introduces you to distributed systems and their “what” and “why.”

- We will then discuss the core concepts of distributed systems in detail. We will also provide diagrams and examples wherever necessary.

- The next chapter is dedicated to distributed data, where we discuss the core concepts related to data handling in distributed systems.

- After that, we discuss data-in-motion—in other words, how to efficiently move data inside a distributed system among its many nodes.

- Then we talk about large-scale data processing. This topic could fill multiple courses on its own, but we have tried to provide you with the basic ideas necessary to this course.

- Next we discuss the common architectural patterns of distributed systems. In the industry today, these patterns are used very frequently by established companies. The chapter will help you to build intuition about these patterns.

- Finally, we finish the course by providing many resources for you to explore further. 

Remember, the world of distributed systems is massive. You keep learning and solving problems all the time when you work on systems like this as a part of your job. 

If you want to pursue real-world problem solving, working on distributed systems is totally worth the effort. And hopefully, this course will direct you towards the right approach.

Let's begin!

Get an idea of what to expect from the course.

Introduction

What Distributed Systems Achieve for Us

Data in Distributed Systems

Communication Between Nodes

Data Processing in Large Scale

Distributed System Architectural Patterns

Case Study 1: Apache Spark

Case Study 2: Apache Druid

Conclusion

Welcome to the Course