Shell - (on my way to) Mastering the command line

system variables (aka environment variables + shell variables)

The title of this article : "system variables" is intended as a generic term meaning both the environment variables and the shell variables

There's a subtle difference between them, which is mostly a matter of scope (source) :

  • environment variables are shell variables that are available system-wide because they have been exported.
  • shell variables only exist in a given shell context (such as an interactive shell, or a subshell created while executing a script).
Since any shell variable can become an environment variable once exported, I see no need of listing them in distinct articles.

!$
last argument of the preceding command. Said to be equivalent to $_ (see also)
$!
PID of last job run in background. Try it : pwd & echo $!
$#
number of parameters passed to the current script
$$
PID of current process
  • It _may_ sound like a good idea to use this as part of a temporary file name so that concurrent executions of a script won't collide, but mktemp will do that better.
  • It's very convenient to detect which flavor of shell is being used : ps $$
    PID	TTY	STAT	TIME	COMMAND
    18060	pts/1	Ss	0:01	/bin/bash
$*
all the parameters passed to the current script, as a space-separated string (source) :
  • $* and $@ hold a b c d (4 distinct elements)
  • "$*" holds a b c d (a single string having spaces). This is not iterable because of the explicit double-quotes.
  • "$@" holds a b c d (the 3 script parameters). The double quotes protect spaces when they are part of the parameter itself.
Check it :
#!/usr/bin/env bash
# run :	./script.sh a b 'c d'

echo -e '\nwith $* :'
for i in $*; do echo $i; done

echo -e '\nwith $@ :'
for i in $@; do echo $i; done

echo -e '\nwith "$*" :'
for i in "$*"; do echo $i; done

echo -e '\nwith "$@" :'
for i in "$@"; do echo $i; done
with $* :
a
b
c
d

with $@ :
a
b
c
d

with "$*" :
a b c d

with "$@" :
a
b
c d
$-
current options set for the shell, the doc says it is not completely reliable
$0
name of the shell or current shell script (see also ${BASH_SOURCE[0]})
$?
decimal exit code of previous process (aka "most recent pipeline").
This handles the value returned by return.
$@
all the parameters passed to the current script or function, as separate words. This is an "iterable list of strings" (source, see $* for details)
You should practically always use double quotes around "$@".
$_
  • definition 1 : when Bash invokes an external command, the variable $_ is set to the full path name of the command and passed to that command in its environment.
  • definition 2 : final argument of previous command executed
BASH_SOURCE[0]
this is almost a synonymous of $0 but has slight differences (source) :
${BASH_SOURCE[0]} $0
  • holds the current script name, when executed or sourced
  • is read-only
  • is Bash-specific
  • can be shortened to $BASH_SOURCE (Bash allows referring to the element 0 of an array with the array name itself)
  • holds the current script name only when executed (something else when sourced)
  • can be overwritten
  • is POSIX-compliant
Check it with this script :
#!/usr/bin/env bash
echo -e "
\$0:$0
\$BASH_SOURCE:$BASH_SOURCE
\${BASH_SOURCE[0]}:${BASH_SOURCE[O]}
" | column -s ':' -t
  • ./test.sh
    $0			./test.sh
    $BASH_SOURCE		./test.sh
    ${BASH_SOURCE[0]}	./test.sh
  • source test.sh
    $0			/bin/bash
    $BASH_SOURCE		test.sh
    ${BASH_SOURCE[0]}	test.sh
DISPLAY
  • indicates where the X server is. Format :
    host:displayNumber.screenNumber
    Numbers start at 0 !
  • To temporarily connect to a specific X server, just type (details) :
    command_line -display (or --display ?) host:displayNumber.screenNumber
  • If you're connecting to the remote host via SSH, consider using the X11 forwarding option.
EDITOR
set the program to use to edit :
  • crontabs
  • Git commit messages, rebases, patches, ...
  • files viewed with less
  • ...
http_proxy
Set it with :
export http_proxy=http://host:port
Same goes on for https_proxy and ftp_proxy.
HOME
current user's home directory, and default for the cd builtin command
HOSTNAME
The name of the current host
IFS
input field separator : used to split string into substrings, defaults to "[space][tab][newline]"
LC_NUMERIC
defines whether numbers should be displayed with a point or a comma as a decimal separator
LOGNAME
Depending on the Unix / Linux flavor :
  • either the name of the user that initially logged in, even though this user executed su afterwards (source : 1, 2)
  • or a synonymous for USER
OLDPWD
The previous working directory as set by the cd builtin (see PWD)
PATH
  • If you type PATH=path/to/some/directory, the whole content of the $PATH variable will be overwritten. This variable expects several :-separated entries.
  • To append an entry to $PATH, just type :
    PATH=$PATH:path/to/some/directory
PIPESTATUS
Array holding exit statuses of last executed foreground pipe. Indexes start at 0.
true | true | false | true | false | false | true; echo ${PIPESTATUS[*]}
0 0 1 0 1 1 0
true | true | false | true | false | false | true; echo "${PIPESTATUS[0]} ${PIPESTATUS[1]} ${PIPESTATUS[2]} ${PIPESTATUS[3]} ${PIPESTATUS[4]} ${PIPESTATUS[5]} ${PIPESTATUS[6]}"
0 0 1 0 1 1 0
PPID
The shell's parent process identifier (or parent's pid if you like)
PROMPT_COMMAND
The contents of this variable are executed as a regular Bash command just before Bash displays a prompt.
PS1
is the shell prompt (details)
PWD
The current working directory as set by cd (see OLDPWD). Using currentDir=$(pwd) is double work since PWD is already available.
RANDOM
Returns a pseudorandom integer in the range 0 - 32767.
To roll a dice :
  • echo $[$RANDOM % 6]
  • echo $(($RANDOM % 6))
See also shuf.
SHLVL
number of nested subshells (starts at 1). Try it : echo -n $SHLVL; bash -c 'echo -n $SHLVL; bash -c "echo -n \$SHLVL"; echo -n $SHLVL; exit; '; echo -n $SHLVL will output :
12321
TMPDIR
directory in which Bash creates temporary files for its own use
USER
the current user name, replaces an overkill $(whoami) in scripts

This variable is often set as an environment variable by Bash login startup files, but it is not actually a Bash builtin (source).

WARNING: terminal is not fully functional

Situation :

When running man, less, ..., in some specific conditions (e.g. within an SSH session nested inside a screen session), you may get this error message :
WARNING: terminal is not fully functional
- (press RETURN)

Details :

This is because the TERM environment variable is not / improperly set.

Solution :

export TERM=xterm-256color
Or fix it permanently.

-bash: !whatever: event not found, exclamation marks, events, what are these ?

Long story short :
  1. this has to do with the shell history
  2. history entries are named events
  3. the shell interprets inputs like !something as commands to recall an event (details)

So, if you just want to use an exclamation mark (not followed by a space) in a string, just simple-quote it :

echo Hello world !
Hello world !
echo Hello world ! Hello everybody !
Hello world ! Hello everybody !
echo ah!ah!ah!
bash: !ah!ah!: event not found
echo "ah!ah!ah!"
bash: !ah!ah!: event not found
echo 'ah!ah!ah!'
ah!ah!ah!
echo Hello everybody !!
Surprise... but remember !! can express more than just extreme happiness

Bash shortcut keys

Bash has Emacs (default, see table below) and Vi shortcut sets. To define your favorite shortcut style :
ALT CTRL
a move to the beginning of the line
conflicts with screen's CTRL-a-... shortcuts
c send the SIGINT signal to stop the current process
d delete everything after the cursor exits the current shell
e move to the end of the line
l clear the screen
q resume from CTRL-s
r search history of commands
s pause output to the screen. Useful when running a verbose command :
for i in {1..1000}; do echo $i; sleep 0.1; done
x-x Jump back and forth between the cursor position and the beginning of the line
z send the SIGTSTP signal to the current foreground process to send it to the background. Use fg to bring it back to the foreground.
_ Undo the last keystroke. Can be repeated to undo several keys back
. paste the last argument of the previous command. Repeat to cycle back in previous commands

How to perform search / replace on a specific line of a file ?

Situation :

I have :
testString='I like bananas.\nbananas are great.\nbananas are what I prefer.\nMy favorite fruit is bananas.'; echo -e "$testString"
I like bananas.
bananas are great.
bananas are what I prefer.
My favorite fruit is bananas.
I want :
I like bananas.
bananas are great.
oranges are what I prefer.
My favorite fruit is bananas.

Solution :

With awk (source) :

echo -e "$testString" | awk '/prefer/ { gsub("bananas","oranges",$0); print $0 }'

oranges are what I prefer.

This only returns the processed line. Needs a little more work

With sed :

echo -e "$testString" | sed -r '/prefer/ s/bananas/oranges/'

I like bananas.
bananas are great.
oranges are what I prefer.
My favorite fruit is bananas.

testFile='./testFile'; echo -e "$testString" > "$testFile"; echo 'BEFORE :'; cat "$testFile"; sed -i '/prefer/ s/bananas/oranges/' "$testFile"; echo -e '\nAFTER :'; cat "$testFile"; rm "$testFile"

BEFORE :
I like bananas.
bananas are great.
bananas are what I prefer.
My favorite fruit is bananas.

AFTER :
I like bananas.
bananas are great.
oranges are what I prefer.
My favorite fruit is bananas.

At first sight, sed looked simpler to me than Awk, but Awk is also able to work on a specific field of every line, or on a specific field of a specific line.

Wildcards and quotes

Quoting shell variables is mandatory to protect scripts from unexpected effects should unquoted variables contain SPACE characters (or any other character considered by the shell as a word separator, such as TAB or NEWLINE. Read more about IFS). Indeed, the SPACE character is one of the common argument separators for the shell. Here come the quotes (but they must be set wisely ).
Let's play :
fileName='my file'; touch $fileName; ls -1
file
my
rm $fileName; ls -1
(both gone)
touch "$fileName"; ls -1
my file
rm "$fileName"; ls -1
(gone)
touch ${fileName}; ls -1
file
my
rm ${fileName}; ls -1
(both gone)
touch ${fileName}1 ${fileName}2; ls -1
file1
file2
my
ls -1 $fileName*
file1
file2
my
ls -1 "$fileName*"
ls: cannot access my file*: No such file or directory
rm ${fileName}1 ${fileName}2; ls -1
(all gone)
touch "${fileName}1" "${fileName}2"; ls -1
my file1
my file2
ls -1 $fileName*
no double quotes
ls: cannot access my: No such file or directory
ls: cannot access file*: No such file or directory
ls -1 "$fileName*"
* inside double quotes
ls: cannot access my file*: No such file or directory
ls -1 "$fileName"*
* outside double quotes
my file1
my file2

How commands are read by Bash (source) :

  1. read input
  2. break it up into words and operators, obeying the quoting rules (escape characters, simple and double quotes) :
    • backslash \ : preserve the literal value of the following character (except newline)
    • single quotes '...' : preserve the literal value of each character enclosed within the quotes. A single quote may not occur between single quotes, even when preceded by a \.
    • double quotes "..." : preserve the literal value of all characters enclosed within the quotes, except for $, `...` and \.
    No * within quotes.
  3. perform alias expansion
  4. substitute the tokens into simple and compound commands (e.g. if, for, while, [[, case, ... constructs. More)
  5. perform shell expansions (source) :
    1. brace expansion : {...}
    2. tilde expansion : ~
    3. variables expansion : $myVariable is substituted with its value
    4. commands substitution : $(command) (or `command`) is substituted with its output
    5. arithmetic expansion : $((arithmeticExpression)) is substituted with its result
    6. process substitution : ???
    7. word split : split the result of previous expansions by SPACE (actually, each character of $IFS is a delimiter).
    8. file name expansion : if any word contains *, ? or[, it is considered as a pattern and replaced with an alphabetically sorted list of file names matching the pattern (if any. Otherwise, leave the special character as-is)
  6. redirections, if any
  7. execute the command
  8. wait for the command to complete and collect its exit status

Examples :

Considering we have run : fileName='my file'; touch "${fileName}1" "${fileName}2", (so we actually have files my file1 and my file2), let's list files :

ls $fileName*

  1. variable expansion : ls my file*
  2. word split (already split up) : ls my file*
  3. file name expansion : no file found match the pattern file*, so no change : ls my file*
  4. result

ls "$fileName*"

  1. word/operator breakup : $fileName is recognized as a variable and will be substituted
  2. variable expansion : ls "my file*"
  3. word split : because of the quotes, the [SPACE] between my and file can not be used to split : ls "my file*"
  4. file name expansion : because of the quotes again, the * is taken literally, nothing to expand. No file named exactly my file* (i.e. having a * in the file name) found, so no change : ls "my file*"
  5. result

ls "$fileName"*

  1. word/operator breakup : $fileName is recognized as a variable and will be substituted
  2. variable expansion : ls "my file"*
  3. word split : (as above) nothing to split : ls "my file"*
  4. file name expansion : files my file1 and my file2 are found and substituted : ls my file1 my file2
  5. result

Regular Expressions in shell context

Since I already dealt with that topic earlier and spread information in many places, here's a collection of hyperlinks to the corresponding articles / sections until I clean this up :

On UTF-8-capable systems (and generally speaking : extended charsets), characters lists such as a-z include special characters like à, é or î. Thus, characters lists (a-z) can not be used in regular expressions to discriminate ASCII/non-ASCII characters. To do so, the solution is to build a complete list of all characters to match against with a regular expression : abcdefghijklmnopqrstuvwxyz (source).

Unset variables in Bash / Ksh

For any further reference, here's what happens during an if statement in Bash and in Ksh while the tested variable is actually unset :
Shell Output
if [ ${undefinedVariable} -eq 0 ]; then echo 'this is the "THEN" part'; else echo 'this is the "ELSE" part'; fi
Bash
-bash: [: -eq: unary operator expected
this is the "ELSE" part
Ksh
ksh: [: argument expected
this is the "ELSE" part
if [ ${undefinedVariable} -ne 0 ]; then echo 'this is the "THEN" part'; else echo 'this is the "ELSE" part'; fi
Bash
-bash: [: -ne: unary operator expected
this is the "ELSE" part
Ksh
ksh: [: argument expected
this is the "ELSE" part

We note that :

Heredocs

Send several commands via SSH :

ssh bob@sshServer << EOC
command1
command2
EOC

Send several lines of text to a command via a | :

cat << EOL | grep base
Roses are #ff0000
Violets are #0000ff
All my base are belong to you
EOL

Create a new file :

Step-by-step version :
  1. cat << EOF > newFile.txt
  2. some text, line 1
  3. some text, line 2
  4. some text, line n
  5. EOF
Big-bang version :
cat << EOF > newFile.txt
some text, line 1
some text, line 2
some text, line n
EOF
  • Data is written to newFile.txt
  • EOF is the "end of file" tag (aka stop token). Any string can be used instead.
cat << EOF > newFile.txt
line 1, not indented
    line 2, space-indented
	line 3, TAB-indented
EOF
cat newFile.txt; rm newFile.txt
  • Indentation works only with spaces, not with TAB.
  • The stop token must have no leading whitespace.

Stop token hacks

Single-quoted stop token (source)
This disables :
  • variable expansion (changing $USER into stuart)
  • command substitution (changing $(command) into the result of executing command)
  • arithmetic expansion (changing $((1+1)) into 2)
cat << EOF
current user : $USER
today : $(date +"%a %b %d")
2 apples + 1 banana is $((2+1)) fruits
EOF
current user : stuart
today : Fri Nov 17
2 apples + 1 banana is 3 fruits
cat << 'EOF'
simple quotes : $USER
today : $(date +"%a %b %d")
2 apples + 1 banana is $((2+1)) fruits
EOF
simple quotes : $USER
today : $(date +"%a %b %d")
2 apples + 1 banana is $((2+1)) fruits
Dash (-) -prefixed stop token (source) :

This is a cosmetic hack improving readability of scripts since its allows indenting the heredocs too. The - in the stop token suppresses leading tabs in the output.

  • This has no effect on lines indented with spaces, including the line of the stop token itself which must be TAB-indented. If space-indented, the stop token line becomes invisible which causes an error :
    ./myScript.sh: line 63: warning: here-document at line 23 delimited by end-of-file (wanted `EOF')
    ./myScript.sh: line 64: syntax error: unexpected end of file
  • There must be no space between << and - (Can't explain why, but if you try, it'll fail )
cat << EOF
no indent
 indent one space
	indent one TAB
EOF
no indent
 indent one space
indent one TAB
cat <<-EOF
no indent
 indent one space
	indent one TAB
	 indent one TAB + one space
		indent 2 TAB's
	EOF
no indent
 indent one space
indent one TAB
 indent one TAB + one space
indent 2 TAB's

What if I want to output some special characters without disabling variable expansion ?

value=42; exampleFile='/tmp/myExampleFile.txt'; cat << EOF > "$exampleFile"
if (\$something > $value)
    blah

if (\$anything < ($value/2))
    pooh
else
    nomnom

if (true || false || whatever)
    who_cares
EOF
cat "$exampleFile"; rm "$exampleFile"
outputs :
if ($something > 42)
    blah

if ($anything < (42/2))
    pooh
else
    nomnom

if (true || false || whatever)
    who_cares

How to pipe a multiline heredoc into a command ?

You can do things like :
cat << EOSQL | sqlplus -s / as sysdba | grep -Ev '^$'
SELECT DISTINCT(TRUNC(last_refresh)) FROM dba_snapshot_refresh_times;
query1;
query2;
EOSQL
Or even :
echo -e "query1;\nquery2;" | sqlplus -s / as sysdba | grep -Ev '^$'
But it is simpler to do :
sqlplus -s / as sysdba << EOSQL | grep -Ev '^$'
query1;
query2;
EOSQL
Remember :
  • cat << EOF is fine when redirecting into a file
  • when redirecting to a command, command << EOF looks more appropriate (see useless use of cat)

How to remove the header line from a command output ?

Let's imagine a command (such as a DB query) that outputs something like :
HEADER
data line 1
data line 2
data line 3
You can retrieve all lines except the header ...
... with grep :
echo -e "HEADER\ndata line 1\ndata line 2\ndata line 3" | grep -v 'HEADER'
Requires to know how to match the header line, i.e. knowing HEADER.
... with sed :
  • echo -e "HEADER\ndata line 1\ndata line 2\ndata line 3" | sed -n '2,$ p'
  • or even simpler : echo -e "HEADER\ndata line 1\ndata line 2\ndata line 3" | sed 1d
... with tail :
echo -e "HEADER\ndata line 1\ndata line 2\ndata line 3" | tail -n +2

Why does this {start..stop..step} output {start..stop..step} instead of a sequence of numbers ?

The "step" feature of the brace expansion is a new feature of Bash 4 (source). To get Bash version :

How to display the n leading / trailing characters from each line of a file ?

Leading characters :

Given the data file :
for i in {1..1000}; do echo $RANDOM >> data.txt; done
sed can do it :
sed -r 's/(^.{3}).*$/\1/g' data.txt
But it's overkill as cut can do it way easier :
cut -c -3 data.txt

Trailing characters :

No such option in cut, so let's use sed :
echo hello | sed -r 's/.*(.{3})$/\1/g'

Job control

Job control is nothing but the ability to stop / suspend / resume the execution of processes. A jobId is displayed when starting a process in the background :
user@host $ emacs & vlc &
[1] 10367
[2] 10368
Here, emacs is the 1st command we've launched, 1 is its jobId and 10367 is its PID.

List the current jobs :

jobs
[1]-	Running	emacs &
[2]+	Running	vlc &
jobs -l
[1]-	10367	Running	emacs &
[2]+	10368	Running	vlc &
Field Value Description Example
[n]
Job ID
  • To be used with fg, bg, wait, kill, ...
  • The job ID must be prefixed by a %
+ or -
  • + : current job
  • - : previous job
10367 PID
Job status :
  • Running : currently running (not stopped / suspended)
  • Stopped : job is suspended

The %jobId syntax used to refer to a job is also known as jobspec.

Suspend a running job :

Resume a suspended job :

  • In the foreground : fg %2
  • In the background : bg %5

Curly brackets & shell Brace Expansion

Output a list of values (details) :

myCommand {value1,value2,value3}
is equivalent to :
myCommand value1; myCommand value2; myCommand value3

Output a sequence of characters or numbers (details):

ascending numbers :
echo {2..8} : 2 3 4 5 6 7 8
reverse order letters :
echo {z..a} : z y x w v u t s r q p o n m l k j i h g f e d c b a
descending numbers with leading zeros (source) :
echo {100..00..10} : 100 090 080 070 060 050 040 030 020 010 000

Other brace expansions :

Syntax Description Example
{value1,value2,value_n}
String generation
Generate as many strings as the number of parameters, including a prefix and/or a suffix (examples)
echo foo{1,2,3}bar
foo1bar foo2bar foo3bar
  • {start..stop}
  • {start..stop..step}
Sequence generation
Generate a string for each parameter from the specified interval, including a prefix and/or a suffix
The step parameter appears with Bash 4 (details, examples).
This construct should be preferred to seq because it won't start subprocesses.
echo test_{1..2}{a..b}_
test_1a_ test_1b_ test_2a_ test_2b_
echo {a..z..7}
a h o v
${parameter:-default}
Use default value
If parameter is unset or null, default (which may be an expansion) is substituted. Otherwise, the value of parameter is substituted.
This can be used to make some function parameters optional (by giving them a default value when omitted) :
myFunction() {
	local myVariable=$1
	local myOtherVariable=${2:-42}
	...
	}
key='value'; echo ${key:-'nothing'}; unset key; echo ${key:-'nothing'}
value
nothing
key='value'; default=42; echo ${key:-$default}; unset key; echo ${key:-$default}
value
42
${parameter:=default}
Assign default value
If parameter is unset or null, default (which may be an expansion) is assigned to parameter. The value of parameter is then substituted.
  • if parameter is set : do nothing
  • otherwise (i.e. parameter is unset or null) : assign default to parameter (i.e. alters parameter)
key='value'; result=${key:='nothing'}; echo $key; unset key; result=${key:='nothing'}; echo $key
value
nothing
${parameter:+value}
Use value if parameter exists
If parameter exists, substitute (e.g. return) value (which may be an expansion).
Otherwise (parameter is null or unset), return nothing.
key='value'; echo ${key:+'key exists'}; unset key; echo ${key:+'key exists'}
key exists
(empty line)
${parameter?message}
Return parameter, or display message if unset
value=42; echo ${value?this variable is unset.}; unset value; echo ${value?this variable is unset.}
42
bash: value: this variable is unset.
${parameter:offset:length}
Substring Expansion
Expands to up to length characters of parameter starting at the character specified by offset (0-indexed). If :length is omitted, go all the way to the end. If offset is negative (use parentheses!), count backward from the end of parameter instead of forward from the beginning.
The last character of a string : ${parameter:(-1):1}
myString='0123456789'; echo ${myString:2}; echo ${myString:4:2}; echo ${myString:(-4):2};
23456789
45
67
myString='azerty'; echo ${myString:(-1):1}
y
  • ${#myString}
  • ${#myArray}
Number of sub-elements
  • length of the string myString
  • number of items in the array myArray Don't use this!
myString='foo bar'; echo ${#myString}
7
${parameter#pattern}
Remove pattern from the beginning of parameter
The pattern is matched against the beginning of parameter. The result is the expanded value of parameter with the shortest match deleted.
This can be used to retrieve the extension of a file name.
myString='abcdef'; echo ${myString#abc}; echo ${myString#[abc]}; myString='aaabbbccc'; echo ${myString#a*b}
def
bcdef
bbccc
name=file.txt; echo ${name#*.}
txt
${parameter##pattern} As above, but the longest match is deleted.
This can be used to retrieve the current directory name.
myString='aaabbbccc'; echo ${myString##a*b}
ccc
cd /var/log && echo ${PWD##*/}
log
${parameter%pattern}
Remove pattern from the end of parameter
The pattern is matched against the end of parameter. The result is the expanded value of parameter with the shortest match deleted.
This can be used to :
  • retrieve a file name without extension. If the extension is known, basename can do it very easily :
    name=file.txt; echo ${name%.*}
    file
  • retrieve the directory name given an absolute or relative file name :
    for fileName in /absolute/path/to/file relative/path/to/file fileWithoutPath; do echo ${fileName%/*}; done
    /absolute/path/to
    relative/path/to
    fileWithoutPath		ooops : this is not a directory
    Workaround :
    for fileName in /absolute/path/to/file relative/path/to/file fileWithoutPath; do [[ "$fileName" =~ / ]] || fileName="./$fileName"; echo ${fileName%/*}; done
    /absolute/path/to
    relative/path/to
    .			at least we get a directory
  • build a logfile name based on the script name :
    script='/path/to/my script.sh'; logFile=$(basename "$script"); logFile=${logFile%.*}'.log'; echo "$logFile"
    my script.log
myString='aaabbbccc'; echo ${myString%a*b}; echo ${myString%b*c}
aaabbbccc
aaabb
${parameter%%pattern} As above, but the longest match is deleted. myString='aaabbbccc'; echo ${myString%%b*c}
aaa
${parameter/search/replace} Results in the expanded value of parameter with the first match of search replaced by replace. myString='abcd abcd'; echo ${myString/cd/CD}
abCD abcd
${parameter//search/replace} As above, but every match of search is replaced. myString='abcd abcd'; echo ${myString//cd/CD}
abCD abCD
  1. ${parameter^}
  2. ${parameter^^}
  3. ${parameter,}
  4. ${parameter,,}
Return parameter with :
  1. 1st character uppercase
  2. all characters uppercase
  3. 1st character lowercase
  4. all characters lowercase
(source)
myString='hello, world'; echo ${myString^}; echo ${myString^^}; myString=${myString^^}; echo ${myString,}; echo ${myString,,}
Hello, world
HELLO, WORLD
hELLO, WORLD
hello, world

Bash command-line tips

Enable / disable verbose mode (source) :

  • Enable :
    • bash -v
    • set -v
    • set -o verbose
  • Disable :

Replay and amend the previous command (source) :

!! replays the previous command :
$ apt-get install package
$ sudo !!
You can even modify and replay a previous command : !!:s/wrong/right

Re-use a previous (long-typed) argument (source) :

  1. Let's imagine you just typed : anyCommand a/very/long/and/complex/path/to/a/file
  2. To re-use the argument (filename, here) with another command, just type the command and press ALT-. to paste the argument.
  3. Press ALT-. again to scroll back the arguments list.

Edit and replay the previous command (source) :

If you typed a command that failed, such as :
cd /rome
you can fix it with the ^wrongString^rightString^ syntax :
^rome^home^
This will perform strings substitution and execute :
cd /home

Shell exit codes

Code Meaning
0 success
(aka "UNIX_SUCCESS" in my scripts)
1 catchall for general errors
(aka "UNIX_FAILURE" in my scripts)
2 misuse of shell builtins
124 specific case with timeout
126 command invoked cannot execute
127 command not found
128 invalid argument to exit
128 + n fatal error signal n
130 script terminated by CTRL-c
255* exit status out of range

Exit codes over 255

On some special cases (such as programs launching shell commands as a child process), the exit code may be shown on 2 bytes :
Bits Meaning
15-8 shell command (child process) exit code
7 =1 if a core dump was produced
6-0 signal number that killed the process
  • So, given the 32512 exit code, which gives 0111 1111 0000 0000 in binary, we can deduce that the shell exit code was 127.
  • Faster solution : final exit code = exit code modulo 255 :
    • echo $((32512%255))
      127
    • with bc :
      32512%255
      127
      The modulo operator % fails when bc is started with its -l flag (source). Workaround : update aliases or start bc with /usr/bin/bc