Gain insights into data processing using Bash, sed, awk, and RegEx. Explore sorting, searching, replacing, and optimizing data tasks interactively with animated videos, projects, quizzes, and tasks.

tarballBash.tar.gz

code-job

spa-job

code-job-bash

2500+ students have taken this innovative project-based data learning course (includes video lectures). It demonstrates the use of Bash shell (Bash, sed and awk including RegEx) in processing textual data. It can help to learn to sort, search, match, replace, clean and optimize various aspects of data with Bash Shell. The target audience (students, researchers, scientists, journalists, data miners, developers) didn't have to go through any tough learning curve. This course also should have helped RedHat, SuSE and Ubuntu Linux learners and Data Science enthusiasts.  Regularly updated, new projects to come!

- Learn Bash commands interactively
- Projects with own stories and conclusive decisions
- Animated video lectures (for visual learners)
- Demonstrations
- Quizzes
- Learning tasks

Bloom's taxonomy (remember, understand, apply, analyze, evaluate and create) in developing your Linux skills. Learn Scientific Programming! scientificprogramming.io

Learn Data Science with Bash Shell

AWK has the mathematical functions like `exp`, `log`, `sqrt`, `sin`, `cos`, `atan2`, etc. built-in, other built-in functions are:

+ `length` the length of its argument taken as a string, or of` $0` if no argument.
+ `rand` random number between `0` and `1`
+ `srand` sets seed for rand and returns the previous seed.
+ `int` truncates to an integer value
+ `substr(s, m, n)` the `n`-character substring of `s` that begins at position `m` counted from `1`.
+ `index(s, t)` the position in `s` where the string `t` occurs, or `0` if it does not.
+ `match(s, r)` the position in `s` where the regular expression `r` occurs, or `0` if it does not. The variables `RSTART` and `RLENGTH` are set to the position and length of the matched string.
+ `split(s, a, fs)` splits the string `s` into array elements `a[1], a[2], ..., a[n]`, and returns `n`. The separation is done with the regular expression `fs` or with the field separator `FS` if `fs` is not given. An empty string as field separator splits the string into one array element per character.
+ `sub(r, t, s)` substitutes `t` for the first occurrence of the regular expression `r` in the string `s`. If `s` is not given, `$0`  is used.
+ `gsub` same as sub except that all occurrences of the regular expression are replaced; sub and gsub return the number of replacements.
+ `sprintf(fmt, expr, ... )` the string resulting from formatting `expr ...` according to the `printf` format `fmt`.
+ `system(cmd)` executes cmd and returns its exit status.
+ `tolower(str)` returns a copy of str with all upper-case characters translated to their corresponding lower-case equivalents.
+ `toupper(str)` returns a copy of str with all lower-case characters translated to their corresponding upper-case equivalents.

AWK has the mathematical functions like `exp`, `log`, `sqrt`, `sin`, `cos`, `atan2`, etc. built-in, other built-in functions are:

+ `length` the length of its argument taken as a string, or of` ＄0` if no argument.
+ `rand` random number between `0` and `1`
+ `srand` sets seed for rand and returns the previous seed.
+ `int` truncates to an integer value
+ `substr(s, m, n)` the `n`-character substring of `s` that begins at position `m` counted from `1`.
+ `index(s, t)` the position in `s` where the string `t` occurs, or `0` if it does not.
+ `match(s, r)` the position in `s` where the regular expression `r` occurs, or `0` if it does not. The variables `RSTART` and `RLENGTH` are set to the position and length of the matched string.
+ `split(s, a, fs)` splits the string `s` into array elements `a[1], a[2], ..., a[n]`, and returns `n`. The separation is done with the regular expression `fs` or with the field separator `FS` if `fs` is not given. An empty string as field separator splits the string into one array element per character.
+ `sub(r, t, s)` substitutes `t` for the first occurrence of the regular expression `r` in the string `s`. If `s` is not given, `＄0`  is used.
+ `gsub` same as sub except that all occurrences of the regular expression are replaced; sub and gsub return the number of replacements.
+ `sprintf(fmt, expr, ... )` the string resulting from formatting `expr ...` according to the `printf` format `fmt`.
+ `system(cmd)` executes cmd and returns its exit status.
+ `tolower(str)` returns a copy of str with all upper-case characters translated to their corresponding lower-case equivalents.
+ `toupper(str)` returns a copy of str with all lower-case characters translated to their corresponding upper-case equivalents.

Course Introduction

Project 1: Analyzing the 'US News' University Ranking Data

Project 2: Facebook Data Mining

Project 3: Australian Cities Crime Statistics

Project 4: Shakespearean-era plays and poems data mining

Bash Tutorials

REGEX Tutorials

AWK Tutorials

SED, GREP and Find Tutorials

Beyond the Text Files! Enter into the Big Data Landscape - Concepts

Conclusion

AWK built-in functions