PBS 146 of X: Shell Loops (Bash)

At the end of the previous instalment we promised loops and arrays for this instalment, but as I started to write the notes I realised that was a little too ambitious! So, we’ll just explore loops in Bash this time. Mind you, there are four different types of loops on the menu, so we have plenty of cool stuff to learn 🙂

Matching Podcast Episode

Listen along to this instalment on episode 761 of the Chit Chat Across the Pond Podcast.

You can also Download the MP3

Read an unedited, auto-generated transcript: CCATP_2023_03_04

Episode Resources

• The instalment ZIP file — pbs146.zip

Loop-like Commands in Bash

Bash supports four loop-like commands — while, until, for, and one that’s a little different to the rest, select. All four need to be used in conjunction with the do and done commands, and all four support the flow-of-control commands continue & break.

While you can make do with just while loops, the reason the other three exist is to make your code shorter and more readable, so I strongly advise using them when they’re the most appropriate choice!

I use the following guidance when choosing which loop command to use

1. While loops are designed to repeat commands while some condition remains true. E.g. keep trying this command until it succeeds.
2. Until loops are the inverse of while loops, they are designed to repeat commands while some condition remains false. E.g. keep asking for input until you get a valid day of the week.
3. For loops are designed to repeat the same command once with each of a list of values. E.g. print the reverse of each line in this file.
4. Select Loops are designed to implement menu systems — ask the user what they’s like to do, do it, then ask them what they’d like to do next, then do it …

Regardless of which of the commands you use, the basic structure of the loops is the same:

LOOP_COMMAND LOOP_ARGS
do
    # the commands to be looped
done

Where LOOP_COMMAND is one of while, until, for & select, and LOOP_ARGS is the appropriate extra info the chosen loop command needs.

At any point between the do and done commands the continue command will jump to the next iteration of the loop, and the break command will end the loop entirely.

Simple While Loops

Like the if command, the while command expects to be passed a command to execute as the first argument, and any arguments to be passed to that command as subsequent arguments. The exit code will determine whether or not the commands between the do and done commands get executed. If the exit code is success, the content of the loop get executed, and the condition is tested again. If the exit code is success again the loop gets executed again. The loop ends the first time the exit code is not success.

Reading a File’s Contents Line-by-Line

We’ve already seen that the read command can be used to read from the standard input stream, and that that’s usually the keyboard, but it’s not always the keyboard. You may well have heard that when you use the pipe (|) symbol to connect two terminal commands you are redirecting the output of the command on the left to the input of the command on the right. The way that works is that anything written to standard out by the command on the left is added to the standard input before the command on the right is called, so, in effect, it is like the command on the left typed into the command on the right.

We used the read command to read a single line of text from the keyboard, if standard in contains text already, read will take the first line of that text, store it in a variable with the given name (first argument to read), and then exit with an exit code of success. When the input stream is empty it will exit with an exit code of failure.

Finally, by default the read command treats back-slashes in the input stream as escape characters. You almost never want this, so you almost always repress the interpretation of escape characters with the -r flag.

Putting all this together we can create a script that reads your computer’s hosts file (/etc/hosts) one line at a time, ignores empty lines and lines starting with a pound/octothorp symbol (#), and prints out all other lines.

We’ll use the continue command to skip over the lines that should be skipped over. You’ll find the code in the instalment ZIP as pbs146a-while.sh:

#!/usr/bin/env bash

echo "Your computer has the following hosts entries:"

# read the hosts file and pipe it to standard in,
# then loop through it line-by-line
cat /etc/hosts | while read -r hostsEntry
do
    # skip empty lines
    if [[ -z $hostsEntry ]]
    then
        continue
    fi

    # skip lines that start with an octothorp
    if echo "$hostsEntry" | egrep -q '^#'
    then
        continue
    fi

    # if we got here, print the line
    echo "* $hostsEntry"
done

Notice that like when we passed a variable name to the read command when reading from the keyboard, we do not pre-fix the name (hostsEntry) with a $ symbol because $ means ‘the value of’.

Also notice the use of the -z test to check if the line is an empty string.

Another important point to notice is the use of the continue command to jump to the next iteration of the loop without printing anything when a line should be skipped.

Finally, I want to explain the check for lines starting with an octothorp:

echo "$hostsEntry" | egrep -q '^#'

This line takes the value of the variable named hostsEntry and sends it to standard in before calling the egrep command to apply a regular expression to the contents of standard in. The regular expression simply means starts with an octothorp, and the -q flag tells egrep to be quiet, i.e. not to write anything to standard out. We’re using this sequence as the condition for an if statement, so we only care about the exit code generated by egrep, which will be success if standard in matches the regular expression. See Taming the Terminal Part 17 for more on Regular Expressions with egrep

Until Loops

The until command works just like the while command except that the commands between do and done get executed as long as the exit code is not success.

The most common use of until loops is to wait for something to happen, or to keep asking a user for input until they give something cromulent.

As an example, let’s wrote a script that needs the user to enter a valid name, which we’ll define as a one or more letters. The users can pass the name as the first argument, or, they’ll be prompted for it.

You’ll find the code in pbs146b-until.sh:

#!/usr/bin/env bash

# use the first arg as the first guess at the name
name=$1

# keep asking for a valid name until we get one
until echo "$name" | egrep -q '^[[:alpha:]]+$'
do
    read -p "What's your name? " name
done

# print a greeting
echo "Well hello there $name 🙂"

For Loops

The for command requires at least three arguments — a name to use when creating a variable that will exist within the loop, the keyword in and one or more values to loop over as a list of arguments.

The commands between do and done get executed once for each additional argument after in, and their value will be available as a variable with the name passed as the first argument to for.

As a general rule, for loops will split their input on spaces, but when you use shell globs (* expansions etc.) file names with spaces do not get split into multiple values. The file pbs146c-forFiles.sh uses a for loop with a file glob (~/*) to show the files in your home directory:

#!/usr/bin/env bash

echo "The files in your home dir:"
for file in ~/*
do
    echo "* $file"
done

By using the $() operator you can loop over the output of a command, one word at a time. Commands that produce space-delimited output are perfect for this, but man commands are smart enough to escape their output when they detect they’re being used in a shell script. The file pbs146d-forCommand.sh uses a for loop using a space-delimited list from a command.

#!/usr/bin/env bash

echo "You belong to the following groups:"
for group in $(groups)
do
    echo "* $group"
done

Finally, the for loop is often used to loop over a series of numbers, there is actually an optional C-like syntax available, but I find it both needlessly verbose and rather confusing because it does variable name expansion between the (( and )) which means you access the value of variables without the $ operator. If you’re curious, this is what it looks like:

for ((i=1; i<=5; i++))
do
    echo "The value of i is $i"
done

Bash offers a much more concise and clear option with the range operator. The range operator generates sequences of numbers between two inclusive values with the very simple syntax {START..END}. The output is a space-separated list of numbers, i.e. exactly what a for loop needs! Also, it works with characters too!

As we can see in the script pbs146e-forRange.sh, we can use this operator twice to print the valid hexadecimal characters:

#!/usr/bin/env bash

echo "The following Hex characters are valid:"
for char in {0..9} {a..f}
do
    echo "* $char"
done

In Bash version 4 and higher things get even cooler with an option to specify a step value with the syntax {START..END..STEP}, but alas, even the latest macOS ships with Bash version 3 🙁

If you need to do more complex sequences in a cross-platform way, the seq command is your friend. It also generates space-separated sequences of numbers, and it does support custom step values. If you pass seq a single argument it assumes you want to start at 1, if you pass it two it assumes you want to start at the first and end at the last in steps of 1, and if you pass three arguments it assumes the first is the starting number, the second the step, and the third the end number. So we can use seq in the following ways:

As an example, the script pbs146f-forStepRange.sh prints the even numbers from zero to twenty:

#!/usr/bin/env bash

echo "The Even Numbers from zero to twenty:"
for n in $(seq 0 2 20)
do
    echo "* $n"
done

Select Menu Loops

The select command uses the same syntax as the for command, but instead of running the commands between do and done once for each additional argument after in, the user is presented with a menu showing all the values, and asked to enter a number representing their choice, that value will go into the loop variable, and the code will execute. This will continue for ever unless the exit or break commands get executed within the loop.

As a simple example, the script pbs146g-select.sh asks what you’d like until you’re done:

#!/usr/bin/env bash

select desert in pancake waffles popcorn 'enough thanks'
do
    if [[ $desert == 'enough thanks' ]]
    then
        break
    fi
    echo "have some $desert"
done

An Optional Challenge

If you’d like to put your Bash skills to the test, try writing a script that accepts a a whole number as an input, either as the first argument or from a user prompt, then prints out the standard n-times multiplication tables to the screen, i.e., if you give the number 3, the output should be:

1 x 3 = 3
2 x 3 = 6
3 x 3 = 9
4 x 3 = 12
5 x 3 = 15
6 x 3 = 18
7 x 3 = 21
8 x 3 = 24
9 x 3 = 27
10 x 3 = 30

You should use the bc (basic calculator) terminal command to do the arithmetic. You’ll need to teach yourself how to use it either with the help of your favourite search engine, or the man page (man bc).

For bonus credit, update your script to allow the user to specify how high the table should go, defaulting to 10 like above.

Final Thoughts

It’s taken quite a bit of effort during this instalment to come up with looping examples that don’t involve arrays. That’s a pretty good indication of what we should do next! Bash support both regular arrays and what calls associative arrays, which are basically dictionaries. We’ll explore both next time.