Advanced Processing with awk

The awk command

The awk command lets us process text in several ways. We can use it to extract fields in a file, calculate totals, and even change the order of the output of a file. The awk command is a command-line utility. It’s also a scripting language with loops, variables, and conditionals.

Much like sed, awk works slightly differently depending on the version installed on our operating system. On Ubuntu, the awk command that ships out of the box is missing a few features, so we need to install a more full-featured version before moving on:

$ sudo apt install gawk

Let’s explore awk by using some data from the 2010 U.S. Census that shows the population by state. We first create the file population.txt with cat by running the commands on the terminal below.

cat << 'EOF' > population.txt
State,Population,Reps
Alabama,4802982,7
California,37341989,53
Florida,18900773,27
Hawaii,1366862,2
Illinois,12864380,18
New York,19421055,27
South Dakota,819761,1
Wisconsin,5698230,8
Wyoming,568300,1
EOF