Learn about big data and the Hadoop ecosystem, and explore streaming technologies. Gain the essential context to kick-start your big data journey confidently. Perfect for beginners.

Intro to Big Data, Hadoop, and Streaming-01.jpg

hadoop1.tar.gz

HDFS_NAMENODE_USER

HDFS_SECONDARYNAMENODE_USER

HDFS_DATANODE_USER

YARN_RESOURCEMANAGER_USER

YARN_NODEMANAGER_USER

Hadoop

Docker Compose

This course will help introduce new learners  to the field of big data, Hadoop ecosystem, and streaming. Perfect for beginners, this concise course will give you the context you need to start your big data journey on the right foot.

The No Nonsense Introduction To Big Data, Hadoop and Streaming

# Overview of distributed systems

Andrew Tanenbaum, a prominent computer scientist, defines a distributed system as follows:

> A **distributed system** is a collection of independent computers that appear as a single coherent system to its users.

Whenever we refer to the term "computer" in this course, we will mean both a physical computer and a virtual machine.

Consider a physical computer with one or more virtual machines in it (which is the norm lately).

These computers cooperate to achieve a common goal. They don’t need to be located in the same room or building. In fact, they don’t know much about each other.

# Communication

So how do these computers communicate?

They pass messages to each other, just like students did at school using paper pieces, before online messaging became an integral part of our lives.

Distributed system communication is a very complex topic, so let's continue to think of it in this simplified way for the purposes of this course. 

Just remember that all communication methods are based on exchanging messages through a network, and do not rely on the fact that some computers are in the same physical machine, as mentioned above.

In such a case, they might share part of the system’s memory. However, in the distributed paradigm, this is a bad practice and should be avoided at all costs.


Get introduced to Distributed Computing Principles.

Before We Begin

Setting The Stage

The Hadoop Ecosystem

Streaming

Apache Spark

Conclusion

Introduction to the Principles of Distributed Computing

Overview of distributed systems