Home>Courses>Introduction to Big Data and Hadoop

Introduction to Big Data and Hadoop

Delve into Big Data essentials, explore data types, and gain insights into Hadoop components like YARN, MapReduce, HDFS, and Spark. Discover foundations to excel in the growing Big Data field.

Beginner

96 Lessons

10h

Certificate of Completion

Delve into Big Data essentials, explore data types, and gain insights into Hadoop components like YARN, MapReduce, HDFS, and Spark. Discover foundations to excel in the growing Big Data field.

AI-POWERED

Explanations

AI-POWERED

Explanations

This course includes

48 Playgrounds

30 Quizzes

Course Overview

Course Content

Apply Your Skills

Recommendations

Course Overview

This course offers a one-of-a-kind rich and interactive experience to learn the fundamentals and basics of Big Data. Throughout this course, you will have plenty of opportunities to get your hands dirty with functioning Hadoop clusters. You will start off by learning about the rise of Big Data as well as the different types of data like structured, unstructured, and semi-structured data. You will then dive into the fundamentals of Big Data such as YARN (yet another resource manager), MapReduce, HDFS (Hadoo...Show More

This course offers a one-of-a-kind rich and interactive experience to learn the fundamentals and basics of Big Data. Throughout ...Show More

Course Content

1.

Hadoop

5 Lessons

Get familiar with Hadoop’s role in Big Data, its evolution, and core terminologies.

Rise of Big Data

Untitled Masterpiece

Big Data Defined

Big Data vs Data Warehouse

2.

YARN

3 Lessons

Walk through YARN's resource management, workflow, and scheduling for efficient cluster operation.

3.

Map Reduce

12 Lessons

Examine MapReduce's programming model, mapper, reducer, testing, execution, and resiliency in big data.

Testing Reducer

Testing MapReduce Program

Running MapReduce End to End

Exploring MapReduce Runs

Combiner and Partitioner

Putting it Together

4.

HDFS

11 Lessons

Enhance your skills in HDFS architecture, from filesystem fundamentals to practical commands.

The Big Picture

Disk Blocks & HDFS Blocks

Block Replication

Writing and Reading

High Availability

HDFS in Practice

HDFS in Practice II

5.

Spark

11 Lessons

Deepen your knowledge of Spark’s architecture, APIs, RDDs, DataFrames, and execution workflow.

Spark Application Life Cycle

Resilient Distributed Datasets (RDDs)

Running Spark Applications

Anatomy of a Spark Application

Execution of a Spark Application

6.

Input & Output Formats

12 Lessons

Follow the process of exploring input and output format efficiencies through SequenceFile, Avro, and Parquet.

Sequence File: Intro

Sequence File: Reading & Writing

Rows vs Columnar Databases

Avro: Code Generation

Avro: IDL & RPC

Parquet: Definition Level

Parquet: Repetition Level

Parquet: Reading & Writing

Parquet: Projection Schema & Misc. Tools

7.

Misc

5 Lessons

Master the steps to utilizing Zookeeper and Pig for managing distributed systems and parallel data processing.

Zookeeper: Intro

Zookeeper: Example

Zookeeper: Practical

8.

Quiz

6 Lessons

Get familiar with core Big Data and Hadoop concepts through structured quizzes.

9.

Reference: Replication

14 Lessons

Unpack the core of data replication techniques, consistency, latency, and conflict resolution in distributed systems.

Single Leader Replication

Asynchronous VS Synchronous Replication

Followers in Log Replication

Log Replication

Issues in Single Leader Replication

Multi Leader Replication

Issues in Multi Leader Replication

Multi-Leader Topologies

Leaderless Replication

Quorum Variations

Concurrent Writes

Video Streaming Queue a Concurrent Writes Example

10.

Reference: Partitioning

4 Lessons

Explore partitioning strategies to enhance scalability, fault tolerance, and query performance.

Partitioning Schemes

Secondary Indexes

Number of Partitions

11.

Reference: Transactions

9 Lessons

Find out about database transaction concepts and strategies for maintaining data integrity.

Introduction to Transactions

More On Transactions

Isolation Levels

Read Skew and Snapshot Isolation

Concurrent Writes and Lost Updates

Serializability

Two Phase Locking

Serializable Snapshot Isolation

12.

Reference: Issues in Distributed Systems

4 Lessons

Deepen your knowledge of complexities in distributed systems, network issues, and time synchronization.

Agreeing On Time

Working with Time issues

Course Author

Apriori Algorithm for Finding Frequent Itemsets with PySpark

Project

Apriori Algorithm for Finding Frequent Itemsets with PySpark

Apriori Algorithm for Finding Frequent Itemsets with PySpark

Learn to apply the Apriori algorithm using PySpark for efficient analysis of frequent itemsets in retail data.

intermediate

Trusted by 2.5 million developers working at companies

"These are high-quality courses. Trust me. I own around 10 and the price is worth it for the content quality. EducativeInc came at the right time in my career. I'm understanding topics better than with any book or online video tutorial I've done. Truly made for developers. Thanks"

Anthony Walker

@_webarchitect_

"Just finished my first full #ML course: Machine learning for Software Engineers from Educative, Inc. ... Highly recommend!"

Evan Dunbar

ML Engineer

"You guys are the gold standard of crash-courses... Narrow enough that it doesn't need years of study or a full blown book to get the gist, but broad enough that an afternoon of Googling doesn't cut it."

Carlos Matias La Borde

Software Developer

"I spend my days and nights on Educative. It is indispensable. It is such a unique and reader-friendly site"

Souvik Kundu

Front-end Developer

"Your courses are simply awesome, the depth they go into and the breadth of coverage is so good that I don't have to refer to 10 different websites looking for interview topics and content."

Vinay Krishnaiah

Software Developer

"I've tried probably 5-7 different sites and Educative is easily the best. It perfectly blends explanation with interactivity"

Eric Downs

Musician/Entrepeneur

Hands-on Learning Powered by AI

See how Educative uses AI to make your learning more immersive than ever before.

Instant Code Feedback

Evaluate and debug your code with the click of a button. Get real-time feedback on test cases, including time and space complexity of your solutions.

AI-Powered Mock Interviews

Put your skills to the test in a simulated interview setting. Receive personalized feedback based on your performance. Available in Premium & Premium Plus plans.

Adaptive Learning

At various checkpoints throughout Educative courses, you will be prompted to take a quick assessment. Receive a condensed curriculum tailored to your strengths and skill gaps.

Explain with AI

Select any text within any Educative course, and get an instant explanation — without ever leaving your browser.

AI Code Mentor

AI Code Mentor helps you quickly identify errors in your code, learn from your mistakes, and nudge you in the right direction — just like a 1:1 tutor!

Recommended before starting this course

Course

Learn Data Science with Bash Shell

Learn Data Science with Bash Shell

Gain insights into data processing using Bash, sed, awk, and RegEx. Explore sorting, searching, replacing, and optimizing data tasks interactively with animated videos, projects, quizzes, and tasks.

beginner

Learn Data Science with Bash Shell

Related Courses and Skill Paths

Course

Learn Data Science with Bash Shell

Learn Data Science with Bash Shell

Gain insights into data processing using Bash, sed, awk, and RegEx. Explore sorting, searching, replacing, and optimizing data tasks interactively with animated videos, projects, quizzes, and tasks.

beginner

Learn Data Science with Bash Shell

Free Resources

FOR TEAMS

Interested in this course for your business or team?

Unlock this course (and 1,000+ more) for your entire org with DevPath

Learn in-demand tech skills in half the time

Copyright ©2024 Educative, Inc. All rights reserved.

soc2