0% completed

All LessonsFree Lessons (6)

Course Introduction

Introduction What is Bash?

Project 1: Analyzing the 'US News' University Ranking Data

Unirank: Data Preview (head, pipe and csvlook)Find the colleges in the ranklist (grep, pipe and wc)Finding the percent of colleges in the ranklist (wc, grep)Listing the Institutes from a given state (grep)Finding the number of Institutes from each state Finding the correlation between university tuition and ranks Project demonstration Project Summary Test your skills

Project 2: Facebook Data Mining

FB: Data Preview (csvstat, csvcut, head, cat)Finding the number of status entry (csvcut, sort, uniq)Find the most popular status entry (awk, sort, cat, csvcut, head)Finding the most popular entry (Bash function)Project demonstration Project Summary Test your skills

Project 3: Australian Cities Crime Statistics

AU Crime Stats: Introduction Data Preview Row and Columns Statistics (sed, wc, head, cat)Finding the top most crime in the whole country (sort, tail, awk)Finding the top most crime per city (csvcut, sort, tail, awk)Finding the best city in Australia! (Bash Shell Script)Prject demonstration Project Summary Test your skills

Project 4: Shakespearean-era plays and poems data mining

Plays and Poems: Data Preview (cat, cut, head and csvlook)Counting the Plays and Poems (head, sed, grep, wc)Finding plays/poems by each author (sed, sort, uniq, head)Finding the most frequent words by Shakespeare Project Summary

Bash Tutorials

Hello! Bash Bash variables and functions Bash command execution and passing arguments Bash meta characters Bash quotation basics Bash read and store user input Bash redirections Bash conditional statements Bash `loop` statements Bash arithmetic Bash arrays

REGEX Tutorials

Hello! RegEx Basic Regular Expressions (BRE)Extended Regular Expressions (ERE)REGEX character classes REGEX look arounds REGEX atomic groups `(?>)`How to use REGEX in Bash shell scripting?

AWK Tutorials

Hello! AWK AWK Built-in Variables AWK built-in functions AWK useful examples

SED, GREP and Find Tutorials

Hello! SED - Stream Editor SED substitutions SED and regular expressions SED delete, print and grouping Hello! GREP Hello! Find command

Beyond the Text Files! Enter into the Big Data Landscape - Concepts

HDFS - Hadoop Distributed File System

YARN - Yet Another Resource Negotiator!

Big Data file formats

Conclusion

Conclusion and References

Learn Data Science with Bash Shell/

...

/

Spark

Spark

We'll cover the following...

Apache Spark is one of the most recent and the quickest developing projects in the big data space. Its design is intended to offer solution to similar problems as Hadoop. It additionally provides similar advantages, including Distributed computing, higher level interfaces, and Fault Tolerance. Spark separates itself in different significant ways:

...