BASH - The Bourne-Again shell

How to amend and replay the previous command ?

With !! :

!! replays the previous command :
$ apt-get install package
$ sudo !!
To amend and replay a previous command :
!!:s/wrong/right

With ^wrong^right^ :

  1. ls /rome
    ls: cannot access '/rome': No such file or directory
    Ooops
  2. Fix it with the ^wrongString^rightString^ syntax :
    ^rome^home^
    This performs strings substitution and executes :
    cd /home
    ls /home
    lost+found  bob  kevin  stuart

Control Bash history with HISTCONTROL

HISTCONTROL=ignoredups
Log only once repeated commands
HISTCONTROL=ignorespace
Don't log commands prefixed with a space
HISTCONTROL=ignoreboth
Equivalent to HISTCONTROL=ignoredups:ignorespace

HISTCONTROL value is a colon (:) -separated list.

How to customize the shell prompt ?

Foreword :

To proceed :
  1. open ~/.bashrc in your favorite text editor
  2. make changes, save and exit
  3. load changes :
    source ~/.bashrc

A basic colorless prompt : stuart@myWorkstation:~[0]$ :

export PS1='\u@\h:\w[\j]\$ '
Flag Usage
\u userName
\h hostname (up to first .)
\w current working directory
\j number of shell children jobs

Add colors + a green / red square indicating the status of the previous command (more about colors) : stuart@myWorkstation [0] ~$ :

NOCOLOR="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[0;32m\]"
YELLOW="\[\e[1;33m\]"
BLUE="\[\e[1;34m\]"
SQUARE="\342\226\210"

export PS1="\`if [ \$? = 0 ]; then echo '${GREEN}'; else echo '${RED}'; fi\`$SQUARE $RED\u$NOCOLOR@$BLUE\h $NOCOLOR[$YELLOW\j$NOCOLOR] \w\$ "

Improvement : also display the current Git branch (if any) : stuart@myWorkstation [0][master] ~$ :

NOCOLOR="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[0;32m\]"
YELLOW="\[\e[1;33m\]"
BLUE="\[\e[1;34m\]"
SQUARE="\342\226\210"

export PS1="\`if [ \$? = 0 ]; then echo '${GREEN}'; else echo '${RED}'; fi\`$SQUARE $RED\u$NOCOLOR@$BLUE\h $NOCOLOR[$YELLOW\j$NOCOLOR]\`git branch &>/dev/null; if [ \$? ]; then git branch 2>/dev/null | awk '/^\*/ {print \"[$BLUE\"\$2\"$NOCOLOR]\"}'; fi\` \w\$ "
  • the final \$ is an actual $ displayed as part of the prompt
  • to be dynamic, $PS1 requires to embed code, so that it is executed again each time $PS1 is displayed. Variables defined outside of the prompt definition actually are constants.
  • I've not been able (so far) to use the $(...) construct rather than `...` for process substitution
  • This runs git branch twice, which I don't like. See below for a fix.

Improvement of this improvement (still gives stuart@myWorkstation [0][master] ~$ ) :

NOCOLOR="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[0;32m\]"
YELLOW="\[\e[1;33m\]"
BLUE="\[\e[1;34m\]"
SQUARE="\342\226\210"

PROMPT_COMMAND='[ $? = 0 ] && squareColor="$GREEN" || squareColor="$RED"; \
currentGitBranch=$(git branch 2>/dev/null | awk '"'"'/^\*/ {print $2}'"'"'); \
[ -n "$currentGitBranch" ] && displayBranch="[$BLUE$currentGitBranch$NOCOLOR]" || displayBranch=''; \
PS1="$squareColor$SQUARE $RED\u$NOCOLOR@$BLUE\h $NOCOLOR[$YELLOW\j$NOCOLOR]$displayBranch \w\$ "'

Fix of the improvement of this improvement : (virtualenv) stuart@myWorkstation [0][master] ~$ :

NOCOLOR="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[0;32m\]"
brightGreen="\[\e[1;32m\]"
YELLOW="\[\e[1;33m\]"
BLUE="\[\e[1;34m\]"
lightPurple="\[\e[1;35m\]"
SQUARE="\342\226\210"

PROMPT_COMMAND='[ $? = 0 ] && squareColor="$GREEN" || squareColor="$RED"; \
[ -z "$VIRTUAL_ENV" ] && displayVirtualEnv='' || displayVirtualEnv="($brightGreen$(basename "$VIRTUAL_ENV")$NOCOLOR) "
currentGitBranch=$(git branch 2>/dev/null | awk '"'"'/^\*/ {print $2}'"'"'); \
[ -n "$currentGitBranch" ] && displayBranch="[$lightPurple$currentGitBranch$NOCOLOR]" || displayBranch=''; \
PS1="$displayVirtualEnv$squareColor$SQUARE $RED\u$NOCOLOR@$BLUE\h $NOCOLOR[$YELLOW\j$NOCOLOR]$displayBranch \w\$ "'
$VIRTUAL_ENV is defined in virtualenvironmentBaseDir/bin/activate

How to display colors in Bash ?

Displaying colors is as much a feature of the shell as of the terminal running the shell.

List of color codes :

color codes are (mostly) like : 0;xx = dark, 1;xx = light
Black       0;30		White         1;37
Dark Gray   1;30		Light Gray    0;37
Red         0;31		Light Red     1;31
Green       0;32		Light Green   1;32
Brown       0;33		Yellow        1;33
Blue        0;34		Light Blue    1;34
Purple      0;35		Light Purple  1;35
Cyan        0;36		Light Cyan    1;36

Demo : display all available colors :

for lightOrDark in 0 1; do for colorCode in {30..36}; do echo -e "\e[${lightOrDark};${colorCode}m${lightOrDark};${colorCode}"; done; done

Bash wildcards and patterns

Usage :

Bash has a built-in pattern matching parser (which must not be mistaken with actual regular expressions). Wildcards can be used with any command that accepts file names as arguments (i.e. almost anything ).

When receiving a command :
  1. Bash searches for wildcards substitutions to perform on file names : are there any existing files matching the wildcards ?
    • yes : perform substitutions
    • no : leave the command as-is
  2. execute the command obtained from the step above

Wildcards / patterns :

The period "." is NOT a wildcard in Bash.
Wildcard replaced by
~ the path to the home directory of the current user
If the ~ character is quoted, it will NOT be substituted with its value. Check :
echo ~ '~' "~"
will return :
/home/bob ~ ~
It can be more convenient to use $HOME instead.
? any single character
* any sequence of characters (including the empty string)
[abcde]
[a-e]
[!abcde]
[!a-e]
exactly one character from the list : abcde (see examples below)
exactly one character from the range : "a" to "e"
any single character that is not listed : abcde
any single character that is not within the range : "a" to "e"
{foo,bar} exactly one entire word in the options given
`myCommand` anything between the backticks must be considered as a command
Even though this works fine, it is now old-fashioned, and the $(myCommand) construct should be preferred.
?(-)x The regular expression equivalent would be : -?x, i.e. an x preceded by an optional - (requires extglob)

Example :

Command with a wildcard Returns
ls *
echo *
all non-hidden files from the current directory
ls *n
echo *n
all non-hidden files which name ends with a n
ls *n* all non-hidden files which name contains a n, even as the last character
ls *.* all non-hidden files which name contains a ., even as the last character, but NOT as the first character as this would be an hidden file
ls *foo* all non-hidden files which name contains foo, or just displays ls : can not access *foo* : no file or directory of this type if no such file exists
echo *foo* all non-hidden files which name contains foo, or just displays *foo* if no such file exists (no wildcard substitution made before "echoing")
ls ?[ae]* all non-hidden files which name contains either a a or a e as the 2nd character
touch foo[123]; ls foo[123] : no matching file found, so no substitution possible : Bash took it literally
echo BEFORE; touch foo1 foo2 foo3; ls; echo AFTER; rm foo[123]; ls
BEFORE
foo1  foo2  foo3
AFTER
(void)
files matching the wildcard exist, so Bash made the corresponding substitutions before running rm

Bash (non-)?(login|interactive) shells

Different flavors of shell :

Shell type Description Use case
login This is a shell that is opened after authenticating, either locally (e.g. at boot time) or via SSH : the prompt is displayed after a successful login (going through /bin/login). The login shell is the first process that executes under your user ID when you log in for an interactive session. virtual terminal (Ctrl-Alt-Fn), SSH, ...
non-login The prompt is displayed without re-authenticating. Opening a new shell emulator such as gnome-terminal
interactive The shell waits for keyboard input to execute commands. virtual terminal (Ctrl-Alt-Fn), SSH, emulator such as gnome-terminal or xterm, ...
non-interactive A non-interactive shell is usually present when a shell script is running. It is non-interactive because it is processing a script and not waiting for user input between commands. For these shell invocations, only the environment inherited from the parent shell is used.
Such shells are also used by daemons / "rc scripts".
And because an image is worth 1000 words (full-size version) :

Files read :

  • interactive login shell :
    1. /etc/profile
    2. ~/.bash_profile
    3. ~/.bash_login
    4. ~/.profile
  • interactive non-login shell :
    1. /etc/bash.bashrc
    2. ~/.bashrc
  • non-interactive login shell (daemons only) :
    1. None. Inherits from its parent shell
  • non-interactive non-login shell :
    1. Such shells don't exist (?).

How to set environment variables for a specific daemon ?

This applies specifically to Shinken not being able to send requests outside of our network because of proxy settings : the variables set in the ~/.* files above simply add no effect
Finally, the QnDsolution was to add to /etc/default/shinken (in the top of the file, after the heading comments) :
export http_proxy="http://kevin:password@proxyHost:port/"
export https_proxy="http://kevin:password@proxyHost:port/"

Other version / doesn't agree (!)

Upon user login, the OS starts a shell. Bourne shells read commands from ~/.profile when invoked as the login shell. Bash (which is a Bourne shell) reads commands from ~/.bash_profile when invoked as the login shell. If ~/.bash_profile doesn’t exist, it reads from ~/.profile instead.

A shell launched at any other time (terminal emulator within GUI environment, or through a SSH connection) :

  • is NOT a login shell but an interactive shell
  • doesn't read ~/.profile or ~/.bash_profile
  • reads ~/.bashrc

Therefore :

  • ~/.profile is the place to put stuff that applies to your whole session, such as programs that you want to start when you log in (but not graphical programs, they go into a different file), and environment variable definitions.
  • ~/.bashrc is the place to put stuff that applies only to bash itself, such as alias and function definitions, shell options, and prompt settings. (You could also put key bindings there, but for bash they normally go into ~/.inputrc.)
  • ~/.bash_profile can be used instead of ~/.profile, but you also need to include ~/.bashrc if the shell is interactive. I recommend the following contents in ~/.bash_profile :
    if [ -r ~/.profile ]; then . ~/.profile; fi
    case "$-" in *i*) if [ -r ~/.bashrc ]; then . ~/.bashrc; fi;; esac

~/.bashrc is not executed when opening a new SSH session

When opening a new connection :
  1. If ~/.bash_profile exists, it's executed, then "STOP". If it doesn't exist, go on to the next step.
  2. if ~/.profile exists, it's executed, then "STOP". If it doesn't exist, go on to the next step.
  3. if ~/.bashrc exists, it's executed, then "STOP".

~/.bash_profile or ~/.profile must end on :

. ~/.bashrc

When Bash outputs : -bash: /bin/ls: Argument list too long

Situation :

When a directory contains numerous files, trying to ls, cp, mv, rm these files leads to a Bash error : Argument list too long

Solution :

To take action on these files with Bash, use a construct such as :

for i in *tmp; do command $i; done

Alternate solution :