BASH - The Bourne-Again shell

Bash wildcards and patterns

Usage :

Bash has a built-in pattern matching parser (which must not be mistaken with actual regular expressions). Wildcards can be used with any command that accepts file names as arguments (i.e. almost anything ).

When receiving a command :
  1. Bash searches for wildcards substitutions to perform on file names : are there any existing files matching the wildcards ?
    • yes : perform substitutions
    • no : leave the command as-is
  2. execute the command obtained from the step above

Wildcards / patterns :

The period "." is NOT a wildcard in Bash.
Wildcard replaced by
~ the path to the home directory of the current user
If the ~ character is quoted, it will NOT be substituted with its value. Check :
echo ~ '~' "~"
will return :
/home/bob ~ ~
It can be more convenient to use $HOME instead.
? any single character
* any sequence of characters (including the empty string)
[abcde]
[a-e]
[!abcde]
[!a-e]
exactly one character from the list : abcde (see examples below)
exactly one character from the range : "a" to "e"
any single character that is not listed : abcde
any single character that is not within the range : "a" to "e"
{foo,bar} exactly one entire word in the options given
`myCommand` anything between the backticks must be considered as a command
Even though this works fine, it is now old-fashioned, and the $(myCommand) construct should be preferred.
?(-)x The regular expression equivalent would be : -?x, i.e. an x preceded by an optional - (requires extglob)

Example :

Command with a wildcard Returns
ls *
echo *
all non-hidden files from the current directory
ls *n
echo *n
all non-hidden files which name ends with a n
ls *n* all non-hidden files which name contains a n, even as the last character
ls *.* all non-hidden files which name contains a ., even as the last character, but NOT as the first character as this would be an hidden file
ls *foo* all non-hidden files which name contains foo, or just displays ls : can not access *foo* : no file or directory of this type if no such file exists
echo *foo* all non-hidden files which name contains foo, or just displays *foo* if no such file exists (no wildcard substitution made before "echoing")
ls ?[ae]* all non-hidden files which name contains either a a or a e as the 2nd character
touch foo[123]; ls foo[123] : no matching file found, so no substitution possible : Bash took it literally
echo BEFORE; touch foo1 foo2 foo3; ls; echo AFTER; rm foo[123]; ls
BEFORE
foo1  foo2  foo3
AFTER
(void)
files matching the wildcard exist, so Bash made the corresponding substitutions before running rm

Bash (non-)?(login|interactive) shells

Different flavors of shell :

Shell type Description Use case
login This is a shell that is opened after authenticating, either locally (e.g. at boot time) or via SSH : the prompt is displayed after a successful login (going through /bin/login). The login shell is the first process that executes under your user ID when you log in for an interactive session. virtual terminal (Ctrl-Alt-Fn), SSH, ...
non-login The prompt is displayed without re-authenticating. Opening a new shell emulator such as gnome-terminal
interactive The shell waits for keyboard input to execute commands. virtual terminal (Ctrl-Alt-Fn), SSH, emulator such as gnome-terminal or xterm, ...
non-interactive A non-interactive shell is usually present when a shell script is running. It is non-interactive because it is processing a script and not waiting for user input between commands. For these shell invocations, only the environment inherited from the parent shell is used.
Such shells are also used by daemons / "rc scripts".
And because an image is worth 1000 words (full-size version) :

Files read :

  • interactive login shell :
    1. /etc/profile
    2. ~/.bash_profile
    3. ~/.bash_login
    4. ~/.profile
  • interactive non-login shell :
    1. /etc/bash.bashrc
    2. ~/.bashrc
  • non-interactive login shell (daemons only) :
    1. None. Inherits from its parent shell
  • non-interactive non-login shell :
    1. Such shells don't exist (?).

How to set environment variables for a specific daemon ?

This applies specifically to Shinken not being able to send requests outside of our network because of proxy settings : the variables set in the ~/.* files above simply add no effect
Finally, the QnDsolution was to add to /etc/default/shinken (in the top of the file, after the heading comments) :
export http_proxy="http://kevin:password@proxyHost:port/"
export https_proxy="http://kevin:password@proxyHost:port/"

Other version / doesn't agree (!)

Upon user login, the OS starts a shell. Bourne shells read commands from ~/.profile when invoked as the login shell. Bash (which is a Bourne shell) reads commands from ~/.bash_profile when invoked as the login shell. If ~/.bash_profile doesn’t exist, it reads from ~/.profile instead.

A shell launched at any other time (terminal emulator within GUI environment, or through a SSH connection) :

  • is NOT a login shell but an interactive shell
  • doesn't read ~/.profile or ~/.bash_profile
  • reads ~/.bashrc

Therefore :

  • ~/.profile is the place to put stuff that applies to your whole session, such as programs that you want to start when you log in (but not graphical programs, they go into a different file), and environment variable definitions.
  • ~/.bashrc is the place to put stuff that applies only to bash itself, such as alias and function definitions, shell options, and prompt settings. (You could also put key bindings there, but for bash they normally go into ~/.inputrc.)
  • ~/.bash_profile can be used instead of ~/.profile, but you also need to include ~/.bashrc if the shell is interactive. I recommend the following contents in ~/.bash_profile :
    if [ -r ~/.profile ]; then . ~/.profile; fi
    case "$-" in *i*) if [ -r ~/.bashrc ]; then . ~/.bashrc; fi;; esac

Customize ~/.bashrc

Customize the shell prompt :

  1. add into ~/.bashrc :
    export PS1='\u@\h:\w[\j]\$ '
    Flag Usage
    \u userName
    \h hostname (up to first .)
    \w current working directory
    \j number of shell children jobs
  2. source ~/.bashrc
Add a colored square indicating the status of the previous command (color codes : 1, 2) :
NOCOLOR="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[0;32m\]"
YELLOW="\[\e[1;33m\]"
BLUE="\[\e[1;34m\]"
SQUARE="\342\226\210"

export PS1="\`if [ \$? = 0 ]; then echo '${GREEN}'; else echo '${RED}'; fi\`$SQUARE $RED\u$NOCOLOR@$BLUE\h $NOCOLOR[$YELLOW\j$NOCOLOR] \w\$ "
Improvement : also display the current Git branch (if any) :
NOCOLOR="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[0;32m\]"
YELLOW="\[\e[1;33m\]"
BLUE="\[\e[1;34m\]"
SQUARE="\342\226\210"

export PS1="\`if [ \$? = 0 ]; then echo '${GREEN}'; else echo '${RED}'; fi\`$SQUARE $RED\u$NOCOLOR@$BLUE\h $NOCOLOR[$YELLOW\j$NOCOLOR]\`git branch &>/dev/null; if [ \$? ]; then git branch 2>/dev/null | awk '/^\*/ {print \"[$BLUE\"\$2\"$NOCOLOR]\"}'; fi\` \w\$ "
my notes at that time :
# NB :
#	- the final '\$' is an actual '$' displayed as part of the prompt
#	- to be dynamic, "$PS1" requires to embed code, so that it is executed again each time "$PS1" is displayed. Variables defined outside of the prompt definition actually are constants.
#	- I've not been able (so far) to use the "$(...)" construct rather than "`...`" for process substitution :-(
#	- the 'git branch' part is not optimal because 'git branch' is actually run twice.
This runs git branch twice, which I don't like. I'll try to fix this.
Improvement of this improvement :
NOCOLOR="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[0;32m\]"
YELLOW="\[\e[1;33m\]"
BLUE="\[\e[1;34m\]"
SQUARE="\342\226\210"

PROMPT_COMMAND='[ $? = 0 ] && squareColor="$GREEN" || squareColor="$RED"; \
currentGitBranch=$(git branch 2>/dev/null | awk '"'"'/^\*/ {print $2}'"'"'); \
[ -n "$currentGitBranch" ] && displayBranch="[$BLUE$currentGitBranch$NOCOLOR]" || displayBranch=''; \
PS1="$squareColor$SQUARE $RED\u$NOCOLOR@$BLUE\h $NOCOLOR[$YELLOW\j$NOCOLOR]$displayBranch \w\$ "'

Change the default files permissions (set a creation mask) :

  1. append to ~/.bashrc :
    umask 077
  2. reload settings :
    source ~/.bashrc

History control with HISTCONTROL :

HISTCONTROL=ignoredups
Log only once repeated commands
HISTCONTROL=ignorespace
Don't log commands prefixed with a space
HISTCONTROL=ignoreboth
Equivalent to HISTCONTROL=ignoredups:ignorespace

This is a colon (:) -separated list.

Make shopt settings permanent :

shopt -s autocd
shopt -s cdspell
shopt -s dirspell

~/.bashrc is not executed when opening a new SSH session

When opening a new connection :
  1. If ~/.bash_profile exists, it's executed, then "STOP". If it doesn't exist, go on to the next step.
  2. if ~/.profile exists, it's executed, then "STOP". If it doesn't exist, go on to the next step.
  3. if ~/.bashrc exists, it's executed, then "STOP".

~/.bash_profile or ~/.profile must end on :

. ~/.bashrc

When Bash outputs : -bash: /bin/ls: Argument list too long

Situation :

When a directory contains numerous files, trying to ls, cp, mv, rm these files leads to a Bash error : Argument list too long

Solution :

To take action on these files with Bash, use a construct such as :

for i in *tmp; do command $i; done

Alternate solution :