0% completed

All LessonsFree Lessons (6)

Course Introduction

Introduction What is Bash?

Project 1: Analyzing the 'US News' University Ranking Data

Unirank: Data Preview (head, pipe and csvlook)Find the colleges in the ranklist (grep, pipe and wc)Finding the percent of colleges in the ranklist (wc, grep)Listing the Institutes from a given state (grep)Finding the number of Institutes from each state Finding the correlation between university tuition and ranks Project demonstration Project Summary Test your skills

Project 2: Facebook Data Mining

FB: Data Preview (csvstat, csvcut, head, cat)Finding the number of status entry (csvcut, sort, uniq)Find the most popular status entry (awk, sort, cat, csvcut, head)Finding the most popular entry (Bash function)Project demonstration Project Summary Test your skills

Project 3: Australian Cities Crime Statistics

AU Crime Stats: Introduction Data Preview Row and Columns Statistics (sed, wc, head, cat)Finding the top most crime in the whole country (sort, tail, awk)Finding the top most crime per city (csvcut, sort, tail, awk)Finding the best city in Australia! (Bash Shell Script)Prject demonstration Project Summary Test your skills

Project 4: Shakespearean-era plays and poems data mining

Plays and Poems: Data Preview (cat, cut, head and csvlook)Counting the Plays and Poems (head, sed, grep, wc)Finding plays/poems by each author (sed, sort, uniq, head)Finding the most frequent words by Shakespeare Project Summary

Bash Tutorials

Hello! Bash Bash variables and functions Bash command execution and passing arguments Bash meta characters Bash quotation basics Bash read and store user input Bash redirections Bash conditional statements Bash `loop` statements Bash arithmetic Bash arrays

REGEX Tutorials

Hello! RegEx Basic Regular Expressions (BRE)Extended Regular Expressions (ERE)REGEX character classes REGEX look arounds REGEX atomic groups `(?>)`How to use REGEX in Bash shell scripting?

AWK Tutorials

AWK Built-in Variables

AWK built-in functions

AWK useful examples

SED, GREP and Find Tutorials

Hello! SED - Stream Editor SED substitutions SED and regular expressions SED delete, print and grouping Hello! GREP Hello! Find command

Beyond the Text Files! Enter into the Big Data Landscape - Concepts

HDFS - Hadoop Distributed File System Map Reduce YARN - Yet Another Resource Negotiator!Flume SQOOP Hive Pig Spark HBase Big Data file formats

Conclusion

Conclusion and References

Learn Data Science with Bash Shell/

...

/

Hello! AWK

Hello! AWK

We'll cover the following...

Awk is a powerful tool in the commandline used for processing the rows and columns of a flat texy file. Awk has built in string functions and associative arrays. Awk supports most of the operators, conditional blocks, and loops available in C language. You may want to know, what Awk stands for? It comes from the surnames of its authors “Aho, ...