Bash scripting - Loops

mail

for

With fixed parameters (start, stop, increment) :

With parameters (start, stop, increment) stored in variables :

  • ugly : start=37; stop=73; increment=7; for i in $(eval echo "{$start..$stop..$increment}"); do echo $i; done (source)
  • better : start=37; stop=73; increment=7; for((i=$start; i<$stop; i+=$increment)); do echo $i; done
  • with automatic 0-padding : start=0001; stop=1001; increment=100; for i in $(eval "echo {$start..$stop..$increment}"); do echo $i; done

If you don't just ++ at every loop :

for ((i=1; i<1000000000000000000; i=i*10)); do
echo "$i" | sed -r ': repeat s/([0-9]+)([0-9]{3})(,|$)/\1,\2/; t repeat'
done
This C-style for loop is called an arithmetic for loop (more about for loops).

Looping on items with SPACEs

  • doesn't work :
    list="a b 'c d'"; for item in $list; do echo "$item"; done
    a
    b
    'c
    d'
  • solution :
    list="a|b|c d"; oldIfs=$IFS; IFS='|'; for item in $list; do echo "$item"; done; IFS=$oldIfs
    a
    b
    c d
As a general rule, if you're hitting this kind of behavior :
  • it may be worth considering passing a list of items as a file (one item per line) rather than a string with item separators
  • this file can be easily read with a while read construct
  • people may argue temp files are BAAAD!, but readability matters

for vs while (source) :

  • ugly : for line in $(cat file.txt); do echo $line; done
  • ugly again (UUOC) : cat file.txt | while read line; do echo $line; done
  • better : while read line; do echo $line; done < file.txt (more)
    This is better because you don't need to spawn a sub-process with |, or with $(...), or start the external cat command.
mail

How to retry until success ?

With recursivity :

#!/usr/bin/env bash

doSomething() {
	random=$RANDOM
	echo "I'm working with '$1' and '$2' and '$3' (RANDOM = $random)"
	echo
	[ $random -lt 10000 ] && return 0 || return 1	faking failure
	}

doSomethingUntilSuccess() {
	loopNumber=$((loopNumber+1))
	echo "Doing 'doSomethingUntilSuccess' (loop $loopNumber) ..."
	doSomething 'foo' 'bar' 42 || doSomethingUntilSuccess
	}

loopNumber=0
doSomethingUntilSuccess
echo 'DONE !'
#!/usr/bin/env bash

generateOneWord() {
	local length=$1
	pwgen -0 $length 1
	}

generateWordsUntilMatchRegex() {
	local regex=$1
	echo -n '.'
	generateOneWord "$wordLength" | grep -Ei "$regex" || generateWordsUntilMatchRegex "$regex"
	}

wordLength=40
generateWordsUntilMatchRegex 'h.*e.*l.*l.*o'
generateWordsUntilMatchRegex '^w.*[orl].*d$'

With an until loop :

#!/usr/bin/env bash

trap "{ echo 'CTRL-C detected... Bye-bye.'; exit 1; }" SIGINT

# This function is the job to redo if failed
doSomething() {
	random=$RANDOM
	echo "I'm working with '$1' and '$2' and '$3' (RANDOM = $random)"
	echo
	[ $random -lt 10 ] && return 0 || return 1
	}

loopNumber=1

until doSomething 'foo' 'bar' 42; do
	echo "$loopNumber loops so far, and running ..."
	loopNumber=$((loopNumber+1))
done

echo "DONE !"
echo "In $loopNumber loops "
#!/usr/bin/env bash

doTheJob() {
	echo 'hello world'
	}

checkTheJobIsDone() {
	[ $(($RANDOM % 6)) -ge 5 ]
	}

main() {
	until checkTheJobIsDone; do
		doTheJob
		sleep 1
	done
	echo 'The job is done'
	}

main
mail

while read

Typical case : reading a file line by line :

tmpFile=$(mktemp)
echo -e 'Bob\nKevin\nStuart' > "$tmpFile";
while read name; do
	echo "Hello, '$name' !"
done < "$tmpFile"
rm "$tmpFile"
Hello, 'Bob' !
Hello, 'Kevin' !
Hello, 'Stuart' !

Leave leading and trailing spaces with IFS= :

tmpFile=$(mktemp)
echo -e ' Bob \n Kevin \n Stuart ' > "$tmpFile";
while read name; do
	echo "Hello, '$name' !"
done < "$tmpFile"
while IFS= read name; do
	echo "Hello, '$name' !"
done < "$tmpFile"
rm "$tmpFile"
Hello, 'Bob' !
Hello, 'Kevin' !
Hello, 'Stuart' !
Hello, ' Bob ' !
Hello, ' Kevin ' !
Hello, ' Stuart ' !

Trying to understand the -r :

tmpFile=$(mktemp)
echo -e 'apples\t12\nbananas\t3\ncoconuts\t42' > "$tmpFile";
while read fruit number; do
	echo -e "Number of '$fruit' : '$number'"
done < "$tmpFile"
while read -r fruit number; do
	echo -e "Number of '$fruit' : '$number'"
done < "$tmpFile"
rm "$tmpFile"
Number of 'apples' : '12'
Number of 'bananas' : '3'
Number of 'coconuts' : '42'
Number of 'apples' : '12'
Number of 'bananas' : '3'
Number of 'coconuts' : '42'
Makes no difference (not the right use case ? I'll have to further investigate this one.)

It also works with process substitution :

while read line; do
	echo "$line"
done < <(ps -u $(whoami) | head -10)
Remember : done < <(command)

... and with heredocs too :

while read line; do
	echo "$line"
done <<< $(ps -u $(whoami) | head -10)
Remember : done <<< $(command)
mail

if then else fi

Because I can never remember this construct :
if condition; then
	some
	commands
else
	some
	different
	commands
fi
There's no need for { } surrounding the then and else blocks.
mail

until

Snippets

  • a=5; until [ "$a" -eq 2 ]; do echo $a; a=$((a-1)); done
  • until ping -c 1 192.168.144.118 & >/dev/null; do echo -n '.'; sleep 1; done; echo OK
  • url='http://x.y.z.y/page/content/changing/upon/reload/'
    needle='some text'
    tmpFile=$(mktemp --tmpdir tmp.XXXXXXXX)
    loops=0
    until grep -i "$needle" $tmpFile; do
    	loops=$((loops+1))
    	>$tmpFile
    	wget $url -O $tmpFile
    done
    echo "Number of loops : $loops"

An infinite loop

until false; do date; sleep 1; done
mail

case esac

#!/bin/bash
something='hello world'
case $something in
	h*)
		echo 'foo'
		;;
	*)
		echo 'bar'
		;;
esac
foo
someVariable
  • this is what the various cases will be matched against
  • tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal are performed before matching is attempted
pattern)
  • shell pattern matching (not regexp), do not use quotes (details and examples)
  • tilde expansion, parameter expansion, command substitution, and arithmetic expansion are performed before matching is attempted
  • the final semicolons ;; are mandatory. They can not be omitted to create a "fallback" like in C or with the PHP switch
  • the first pattern that matches is executed, the following ones are skipped
Some of the extended pattern matching operators require the extglob shell option to be enabled (and will produce shell errors otherwise) :
shopt -s extglob
mail

How to interrupt a loop ?

An example is worth 1000 words :
#!/usr/bin/env bash

listOfCommands=': break continue return exit'
for command in $listOfCommands; do
	echo "With '$command'"

	for i in {1..3}; do
		echo -n ' ==> '
		[ "$i" -eq 2 ] && eval "$command"
		echo $i
	done
	echo -e 'This comes after the loop\n'

done
With ':'
 ==> 1
 ==> 2		nothing special, as expected
 ==> 3
This comes after the loop

With 'break'
 ==> 1
 ==> This comes after the loop	stop the whole loop block and continue the script after it

With 'continue'
 ==> 1
 ==>  ==> 3	skip the rest of the current loop and start the next loop
This comes after the loop

With 'return'
 ==> 1
 ==> ./test.sh: line 9: return: can only `return' from a function or sourced script	complain and interrupt nothing
2
 ==> 3
This comes after the loop

With 'exit'
 ==> 1
 ==> 	stop the script, return to the shell prompt