Bash scripting - Loops

mail

for loops

With fixed parameters (start, stop, increment) :

With parameters (start, stop, increment) stored in variables :

  • ugly : start=37; stop=73; increment=7; for i in $(eval echo "{$start..$stop..$increment}"); do echo $i; done (source)
  • better : start=37; stop=73; increment=7; for((i=$start; i<$stop; i+=$increment)); do echo $i; done
  • with automatic 0-padding : start=0001; stop=1001; increment=100; for i in $(eval "echo {$start..$stop..$increment}"); do echo $i; done

If you don't just ++ at every loop :

for ((i=1; i<1000000000000000000; i=i*10)); do
echo "$i" | sed -r ': repeat s/([0-9]+)([0-9]{3})(,|$)/\1,\2/; t repeat'
done
This C-style for loop is called an arithmetic for loop (more about for loops).

Looping on items with SPACEs

  • doesn't work :
    list="a b 'c d'"; for item in $list; do echo "$item"; done
    a
    b
    'c
    d'
  • solution :
    list="a|b|c d"; oldIfs=$IFS; IFS='|'; for item in $list; do echo "$item"; done; IFS=$oldIfs
    a
    b
    c d
As a general rule, if you're hitting this kind of behavior :
  • it may be worth considering passing a list of items as a file (one item per line) rather than a string with item separators
  • this file can be easily read with a while read construct
  • people may argue temp files are BAAAD!, but readability matters

for vs while (source) :

  • ugly : for line in $(cat file.txt); do echo $line; done
  • ugly again (UUOC) : cat file.txt | while read line; do echo $line; done
  • better : while read line; do echo $line; done < file.txt (more)
    This is better because you don't need to spawn a sub-process with |, or with $(...), or start the external cat command.
mail

How to retry until success ?

With recursivity :

#!/usr/bin/env bash

doSomething() {
	random=$RANDOM
	echo "I'm working with '$1' and '$2' and '$3' (RANDOM = $random)"
	echo
	[ $random -lt 10000 ] && return 0 || return 1	faking failure
	}

doSomethingUntilSuccess() {
	loopNumber=$((loopNumber+1))
	echo "Doing 'doSomethingUntilSuccess' (loop $loopNumber) ..."
	doSomething 'foo' 'bar' 42 || doSomethingUntilSuccess
	}

loopNumber=0
doSomethingUntilSuccess
echo 'DONE !'
#!/usr/bin/env bash

generateOneWord() {
	local length=$1
	pwgen -0 $length 1
	}

generateWordsUntilMatchRegex() {
	local regex=$1
	echo -n '.'
	generateOneWord "$wordLength" | grep -Ei "$regex" || generateWordsUntilMatchRegex "$regex"
	}

wordLength=40
generateWordsUntilMatchRegex 'h.*e.*l.*l.*o'
generateWordsUntilMatchRegex '^w.*[orl].*d$'

With an until loop :

#!/usr/bin/env bash

trap "{ echo 'CTRL-C detected... Bye-bye.'; exit 1; }" SIGINT

# This function is the job to redo if failed
doSomething() {
	random=$RANDOM
	echo "I'm working with '$1' and '$2' and '$3' (RANDOM = $random)"
	echo
	[ $random -lt 10 ] && return 0 || return 1
	}

loopNumber=1

until doSomething 'foo' 'bar' 42; do
	echo "$loopNumber loops so far, and running ..."
	loopNumber=$((loopNumber+1))
done

echo "DONE !"
echo "In $loopNumber loops "
#!/usr/bin/env bash

doTheJob() {
	echo 'hello world'
	}

checkTheJobIsDone() {
	[ $(($RANDOM % 6)) -ge 5 ]
	}

main() {
	until checkTheJobIsDone; do
		doTheJob
		sleep 1
	done
	echo 'The job is done'
	}

main
mail

The while read construct

Typical case : reading a file line by line :

tmpFile=$(mktemp)
echo -e 'Bob\nKevin\nStuart' > "$tmpFile";
while read name; do
	echo "Hello, '$name' !"
done < "$tmpFile"
rm "$tmpFile"
Hello, 'Bob' !
Hello, 'Kevin' !
Hello, 'Stuart' !

Leave leading and trailing spaces with IFS= :

tmpFile=$(mktemp)
echo -e ' Bob \n Kevin \n Stuart ' > "$tmpFile";
while read name; do
	echo "Hello, '$name' !"
done < "$tmpFile"
while IFS= read name; do
	echo "Hello, '$name' !"
done < "$tmpFile"
rm "$tmpFile"
Hello, 'Bob' !
Hello, 'Kevin' !
Hello, 'Stuart' !
Hello, ' Bob ' !
Hello, ' Kevin ' !
Hello, ' Stuart ' !

Trying to understand the -r :

tmpFile=$(mktemp)
echo -e 'apples\t12\nbananas\t3\ncoconuts\t42' > "$tmpFile";
while read fruit number; do
	echo -e "Number of '$fruit' : '$number'"
done < "$tmpFile"
while read -r fruit number; do
	echo -e "Number of '$fruit' : '$number'"
done < "$tmpFile"
rm "$tmpFile"
Number of 'apples' : '12'
Number of 'bananas' : '3'
Number of 'coconuts' : '42'
Number of 'apples' : '12'
Number of 'bananas' : '3'
Number of 'coconuts' : '42'
Makes no difference (not the right use case ? I'll have to further investigate this one.)

It also works with process substitution :

while read line; do
	echo "$line"
done < <(ps -u $(whoami) | head -10)
Remember : done < <(command)

... and with heredocs too :

while read line; do
	echo "$line"
done <<< $(ps -u $(whoami) | head -10)
Remember : done <<< $(command)
mail

The if then else fi construct

Because I can never remember this construct :
if condition; then
	some
	commands
else
	some
	different
	commands
fi
There's no need for { } surrounding the then and else blocks.
mail

The until loop

a=5; until [ "$a" -eq 2 ]; do echo $a; a=$((a-1)); done

until ping -c 1 192.168.144.118 & >/dev/null; do echo -n '.'; sleep 1; done; echo OK

url='http://x.y.z.y/page/content/changing/upon/reload/'
needle='some text'
tmpFile=$(mktemp --tmpdir tmp.XXXXXXXX)
loops=0
until grep -i "$needle" $tmpFile; do
	loops=$((loops+1))
	>$tmpFile
	wget $url -O $tmpFile
done
echo "Number of loops : $loops"
mail

The case esac construct

#!/bin/bash
something='hello world'
case $something in
	h*)
		echo 'foo'
		;;
	*)
		echo 'bar'
		;;
esac
foo
someVariable
  • this is what the various cases will be matched against
  • tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal are performed before matching is attempted
pattern)
  • shell pattern matching (not regexp), do not use quotes (details and examples)
  • tilde expansion, parameter expansion, command substitution, and arithmetic expansion are performed before matching is attempted
  • the final semicolons ;; are mandatory. They can not be omitted to create a "fallback" like in C or with the PHP switch
  • the first pattern that matches is executed, the following ones are skipped
Some of the extended pattern matching operators require the extglob shell option to be enabled (and will produce shell errors otherwise) :
shopt -s extglob
mail

How to interrupt a loop ?

An example is worth 1000 words :
#!/usr/bin/env bash

listOfCommands=': break continue return exit'
for command in $listOfCommands; do
	echo "With '$command'"

	for i in {1..3}; do
		echo -n ' ==> '
		[ "$i" -eq 2 ] && eval "$command"
		echo $i
	done
	echo -e 'This comes after the loop\n'

done
With ':'
 ==> 1
 ==> 2		nothing special, as expected
 ==> 3
This comes after the loop

With 'break'
 ==> 1
 ==> This comes after the loop		stop the whole loop block and continue the script after it

With 'continue'
 ==> 1
 ==>  ==> 3		skip the rest of the current loop and start the next loop
This comes after the loop

With 'return'
 ==> 1
 ==> ./test.sh: line 9: return: can only `return' from a function or sourced script		complain and interrupt nothing
2
 ==> 3
This comes after the loop

With 'exit'
 ==> 1
 ==> 		stop the script, return to the shell prompt