Bash Index : G - The 'G' Bash commands : description, flags and examples

mail

getopt

getopt :
mail

groupdel

Usage

Delete a group of users

Example

groupdel someGroup
mail

getopts

Usage

getopts is a shell built-in used to parse positional parameters (details, example)
getopts optstring name [args]
There exists a getopt command (no trailing s) that must not be mistaken for getopts. Watch for typos !

How it works :

  1. Each time it is invoked, getopts places the next option in the shell variable name, initializing name if it does not exist, and the index of the next argument to be processed into the variable OPTIND. OPTIND is initialized to 1 each time the shell or a shell script is invoked.
  2. When an option requires a value, getopts places that value into the variable OPTARG. The shell does not reset OPTIND automatically; it must be manually reset between multiple calls to getopts within the same shell invocation if a new set of parameters is to be used.
  3. When the end of options is encountered, getopts exits with a return value greater than zero.
In other words :
  1. getopts unstacks positional parameters one at a time (which is why it's usually within a while loop).
  2. once processed, each parameter value ends in the name variable, which explains the following case esac construct.
  3. when there are no more parameters to unstack, getopts returns a UNIX_FAILURE status code, causing the end of the while loop.
    This is why getopts can not handle the situation where no argument is passed to a script : since there is nothing to unstack, the script doesn't even enter the while loop.
getopts doesn't support long options (source) :
./myScript.sh --key=value

The option-string (aka optstring, source) :

The optstring tells getopts : 
:fA:x	it's legal to have several : in the optstring
^  ^
|  +---- : following an option : that option requires a value
+------- leading : silent mode
  • which options to expect :
    • fAx means that options -f, -A and -x are expected
  • which options must have a value, by appending them a trailing colon (:) :
    • fA:x means that -A has a value, i.e. the command line will look like myScript.sh -A value
    • This value is mandatory and will cause an error if omitted, unless silenced (see next point).
  • how to handle errors :
    verbose mode silent mode
    enabled with enabled by default a leading colon :fA:x
    $OPTARG on invalid option unset, which means :
    • you have no option to handle this yourself
    • getopts does it for you by displaying an error message like : 
      ./test.sh -z
./test.sh: illegal option -- z
    		 the invalid option character

Example

Read flags only (no value) :

#!/usr/bin/env bash

while getopts ':a' opt; do
	case "$opt" in
		a)	echo '-a was triggered!'		;;
		\?)	echo "Invalid option: -$OPTARG"	;;
	esac
done
./test.sh -z
Invalid option: -z
./test.sh -a
-a was triggered!

Read flags and values :

#!/usr/bin/env bash

while getopts ':a:' opt; do
	case "$opt" in
		a)	echo "Value of option '$opt' : '$OPTARG'"	;;
		\?)	echo "Invalid option: -$OPTARG"			;;
	esac
done
./test.sh -z
Invalid option: -z
./test.sh -a
(nothing)
./test.sh -a 12
Value of option 'a' : '12'
./test.sh -a 12 34
Value of option 'a' : '12'

Detect an option without value (sources : 1, 2) :

#!/usr/bin/env bash

while getopts ':a:' opt; do								same as above
	case "$opt" in
		a)	echo "✅ Option 'a' has value '$OPTARG'"			;;
		:)	echo "⚠️  No value given for option '-$OPTARG'";	exit 1	;;	missing value handled here
		\?)	echo "⛔ Invalid option: '-$OPTARG'";		exit 1	;;
	esac
done

Detect when no option was given :

#!/usr/bin/env bash

options=':ab'

if (! getopts "$options" opt); then			Read more about this below
	echo 'no option given'
	exit 1
fi

while getopts "$options" opt; do
	case "$opt" in
		a)  echo "'-a' was triggered!"			;;
		b)  echo "'-b' was triggered!"			;;
		\?) echo "Invalid option: '-$OPTARG'"; exit 1	;;
	esac
done
The reason for running the first getopts in a subshell is to be able to read the options without interfering with them so that they can be read again in the while loop. If we replace the whole if block with 
getopts "$options" opt || { echo 'no option given'; exit 1; }
it "swallows" the first option : 
./test.sh
no option given			as expected

./test.sh -ab
'-b' was triggered!		where's the a ?

./test.sh -aab
'-a' was triggered!		Houston, we've had a problem 
'-b' was triggered!
mail

gzip

Usage

Compress data
You definitely should have a look at pigz :
  1. apt install pigz
  2. create system-wide aliases : 
    cat << EOF >> /etc/bash.bashrc

alias gzip='pigz'
alias gunzip='unpigz'
EOF
  3. reload aliases :
    . /etc/bash.bashrc

Flags

Flag Usage
-c --stdout --to-stdout Write output on standard output and keep original files unchanged.
gzip someFile
replaces someFile with its compressed counterpart someFile.gz. With -c, you may run :
gzip -c someFile > whatever
to have both the compressed and uncompressed versions of someFile.
gzip stores the original file name into the compressed file so that gunzip can recover it (check it with : file whatever). However, mixing gzip -c and gunzip (without -c) produces unexpected results. 
regularFile=$(mktemp --tmpdir regular.XXXXXXXX); compressedFile=$(mktemp --tmpdir compressed.XXXXXXXX); for i in {1..100}; do echo "$i : Hello world" >> "$regularFile"; done; gzip -c "$regularFile" > "$compressedFile"; ls -lhU "$regularFile" "$compressedFile"; file "$compressedFile"; gunzip "$compressedFile"; rm "$regularFile" "$compressedFile"
-rw------- 1 bob users 1.7K Oct 29 11:18 /tmp/regular.JS4MvZYv
-rw------- 1 bob users	255 Oct 29 11:18 /tmp/compressed.af0tDooE
/tmp/compressed.af0tDooE: gzip compressed data, was "regular.JS4MvZYv", last modified: Sat Oct 29 11:18:41 2016, from Unix
gzip: /tmp/compressed.af0tDooE: unknown suffix -- ignored
--rsyncable Make the compressed file rsync-friendly.
gzip uses adaptative compression methods in which changing a single byte of the source data may cascade many changes in the compressed file. As a result, this would defeat Rsync's "send-only-changed-bytes" strategy. The --rsyncable disables the adaptative compression. (details)
"rsyncable" compressed files are less than 1% larger than non-rsyncable files
gunzip makes no difference between rsyncable / not rsyncable files.
-n
--fast
--best		 Specify the compression level n :
  • -1 --fast : fastest compression (i.e. less compression)
  • -6 : default
  • -9 --best : slowest compression (i.e. best compression)
mail

groupadd

Usage

Create a new group

Flags

Flag Usage
-g gid assign the group ID gid for the new group
-r --system create a system group

Example

groupadd -g 1234 newGroupName
mail

grep

Usage

Return lines matching a pattern, which can be :
The word grep comes from the ed command g/re/p : "global regular expression print".
If grep returns : 
Binary file someFile matches
  • It's pretty likely that someFile has "exotic" characters (i.e. non-ASCII characters such as ISO-8859 or Unicode)
  • The matching lines won't be returned. Consider -a to get them.
(Almost) only grep may return something like : Binary file ... matches. Check this with :
for binaryFile in /bin/* /usr/bin/*; do strings $binaryFile | grep -E 'Binary file.*matches' && echo -e "$binaryFile\n"; done
On my workstation, /bin/grep and usr/bin/emacs (and variants / symlinks) are listed.

Exit Status

Code Condition Details
0 some match was found
1 no match found
2 an error occurred. Except if a match is found while errors are silenced by the quiet mode.
(source)

Flags

Flag Usage
-An --after-context=n display the n lines after the one matching the pattern
-a --text process a binary file as if it were text (see note above)
-Bn --before-context=n display the n lines before the one matching the pattern
-b --byte-offset print the 0-based byte offset within the input file before each line of output
With -o : print the offset of the matching part itself.
-Cn --context=n equivalent to -Bn -An
-c --count print a count of matching lines for each input file. To count all occurrences within input (i.e. several occurrences per line), see this.
--color=value highlight matched string. value can be :
  • auto : when output is redirected, the color information won't be passed
  • always : when output is redirected, the color information will also be passed
  • never : no highlighting
-d action
--directories=action		 if the input is a directory, use action to process it. action can be :
  • skip : (explicit)
  • read : (default) read just as if they were ordinary files (and usually generate an error : grep: myDir: Is a directory)
  • recurse : go down into the directory, similar to -r
-e pattern
  • search multiple patterns :
    seq 1 5 | grep -e '3' -e '5'
    3
5
  • protect patterns starting with a - :
    echo -e 'foo\n-bar\n-baz' | grep -e '-ba'
    -bar
-baz
-E
--extended-regexp 		consider the given pattern as an extended regular expression. grep -E is equivalent to egrep
  • The regular expression may contain a shell variable (use ", no need to escape $) :
    for letter in {a..z}; do echo -e "foo\nbar\nbaz" | grep -E "$letter$letter"; done
  • for readability, it's also possible to store the regular expression itself into an intermediate variable :
    for letter in {a..z}; do regex="$letter{2}\$"; echo -e "foo\nbar\nbaz" | grep -E "$regex"; done
  • grep -E is greedy (i.e. * will match the longest possible string). If you need a lazy star, consider -P :
    string='Hello world'; echo "$string" | grep -E 'e.*o'; echo "$string" | grep -P 'e.*?o'
    Hello world
Hello world
--exclude=pattern
  • skip files matching pattern
  • pattern supports the *, ? and [] wildcards
--exclude-dir=pattern
  • skip directories with name matching pattern
  • with -r : skip sub-directories with name matching pattern
pattern can use *, ?, and [] as wildcards, and \ to quote a wildcard or backslash character literally
-F pattern
--fixed-strings pattern 		Interpret pattern as fixed strings, not regular expressions
makes grep much faster
-f file
--file=file 		get patterns from file, one per line
-G --basic-regexp Interpret the pattern as a basic regular expression. This is the default.
-h --no-filename do not show the name of the file where a match is found. This is the default when grepping a single file
-H --with-filename print the name of the file where a match is found. This is the default when grepping several files
-i --ignore-case ignore case
--include=pattern
  • search only files matching pattern
  • with the same pattern rules than --exclude
-l --files-with-matches show only the name of files having a match. By default, grep outputs the file name and every matched line.
"list matching files"
-L --files-without-match show only the name of files having no match
-m n --max-count=n Stop reading input after n matching lines (example)
grep -m n pattern is a good alternative to grep pattern | head -n
-n --line-number prefix each line of output with the line number within its input file
-o --only-matching show only the part of a matching line that matches the pattern
-P pattern
--perl-regexp pattern 		consider pattern as a Perl regEx
This option is experimental when combined with the -z / --null-data option.
-q --quiet --silent do not write anything to standard output (Read more).
-r --recursive
  • read all files under each directory, recursively
  • does not follow symlinks unless explicitly instructed to do so (see -R)
  • if no file operand is given, grep searches the working directory only
-R like -r but follows symlinks
-s --no-messages suppress error messages about nonexistent or unreadable files
This is fine to workaround the grep: myDir: Is a directory message, but you should consider -d
-v --invert-match invert the sense of matching to select non-matching lines
-w word
--word-regexp word 		return lines containing matches that form whole words. word matches either at the beginning/end of line, and between characters that are not letters, digits or the underscore.
-x
--line-regexp 		return only results that exactly match the whole line. For a regular expression pattern (-E pattern), this is like parenthesizing the pattern and then surrounding it with ^ and $.
echo -e 'foo\nbar\nbaz\nfoobar\nfoobaz\nbarbaz' | grep -x 'bar'
bar
echo -e 'foo\nbar\nbaz\nfoobar\nfoobaz\nbarbaz' | grep -E '^(bar)$'
bar

Example

Elegant hack for grep -v grep (source) :

ps aux | grep '[t]erminal'
This will match lines containing terminal, which grep '[t]erminal' does not.

Search a word in a file, then display 1 line before/after the matching line + line number

grep -A1 -B1 -n -i "expense" report.txt
ignoring case, extract every line from the file reports.txt containing the word "expense", display the lines before and after each match, and print the relevant line numbers.

When there are 1 / n occurrences of pattern per line :

  • count lines having an a
    echo -e "apple\nbanana\ncoconut" | grep -c a
    2
  • same method to (try to) count o
    echo -e "apple\nbanana\ncoconut" | grep -c o
    1
    Because only 1 line has an o (albeit several occurrences within the line)
  • solution
    echo -e "apple\nbanana\ncoconut" | grep -o o | wc -l
    2
  • and it actually returns the total number of occurrences within the input
    echo -e "apple\nbanana\ncoconut" | grep -o a | wc -l
    4

Search and highlight non-ASCII characters within file (source) :

grep --color='auto' -P -n "[\x80-\xFF]" file

Search lines and environment matching a RegEx :

grep --color=auto -B 10 -P "Return code was : [^0][0-9]*" error.log

quiet mode and return codes :

Let's try some grep's :
echo -e 'a\nb\nc' | grep 'b'; echo $?
b
0
echo -e 'a\nb\nc' | grep -q 'b'; echo $?
0
echo -e 'a\nb\nc' | grep 'z'; echo $?
1
echo -e 'a\nb\nc' | grep -q 'z'; echo $?
1
echo -e 'a\nb\nc' | grep ; echo $?
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
2
echo -e 'a\nb\nc' | grep -q; echo $?
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
2
As a summary, both in normal and quiet modes :
  • result found + no error = 0
  • no result found + no error = 1
  • if error : 2
mail

getfacl

Usage

Read the ACL of a file :
getfacl file

Example

getfacl can also be used in the context of a backup/restore of file permissions (source) :

  1. Backup : getfacl -R /path/to/dirContainingFilesWithPermissionsToBackup > /path/to/permissions.txt
  2. Play with file permissions
  3. Restore : setfacl --restore=/path/to/permissions.txt
mail

groups

Usage

Display a user's group memberships

Example

groups bob 
bob : bob adm dialout cdrom plugdev lpadmin admin sambashare
mail

getent

Usage

getent lets admins get entries in a number of important text files called databases. This includes the passwd and group databases which store user information.

Since getent uses the same name service as the system, it will show all information, including that gained from network information sources such as LDAP.

The databases it searches in are :

Exit Status

Code Condition Details
0 Command completed successfully
2 No result found

Example

Get information about a user account :

local account :

getent vs grep /etc/passwd :
  • grep ^root: /etc/passwd
  • getent passwd root
    Both return : 
    root:x:0:0:root:/root:/bin/bash
    • The ^ and : in the grep command are there to make sure we hit the username field of /etc/passwd . getent is much less error-prone.
    • In the getent command line above, passwd refers to the passwd database.
playing with another account :
  • getent passwd mysql
    mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
  • by UID this time :
    getent passwd 27
    mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash

domain account :

As above :
  • getent passwd tanderson
    tanderson:*:1234567890:3456789012:Thomas ANDERSON:/home/tanderson:/bin/bash
  • getent passwd 1234567890
    tanderson:*:1234567890:3456789012:Thomas ANDERSON:/home/tanderson:/bin/bash
This may return no result, mostly because only local accounts are shown for performance reasons. Unless the domain has a huge number of users, it is possible to enable listing all of them (source) :
  1. edit /etc/sssd/sssd.conf
  2. add the line : 
    [domain/myDomain]

enumerate = true
  3. restart sssd :
    systemctl restart sssd

Test whether a group exists :

  • grep ^myGroup: /etc/group
  • getent group myGroup
If myGroup exists, getent returns a Unix success and displays : 
myGroup:x:myGroupId:joe,jack,william,averell
Otherwise, it displays nothing and returns a Unix failure.