Bash Index : P - The 'P' Bash commands : description, flags and examples

mail

pv

Usage

pv (pipe viewer, pv) shows the progress of data through a pipeline by giving information such as :
Even though pv proved useful, it _may_ be deprecated by evolutions of related tools :
  • dd has its own status=progress option
  • rsync too :
    rsync -avz --progress -e ssh path/to/fileToSend bob@destination:/destination/directory
  • scp also displays the progress while transferring :
    scp path/to/fileToSend bob@destination:/destination/directory
    Password: taptaptap
    path/to/fileToSend													30% 2959MB   3.8MB/s   29:23 ETA

Flags

Flag Usage
-b --bytes enable byte counter and display the total amount of data transferred so far
-W --wait wait until the first byte has been transferred before showing any progress information or calculating any ETAs

Example

dd if=/dev/zero | pv | dd of=./zero.txt bs=1 count=10000000
mail

pkill

Usage

pkill [process selection options] [kill options]

Literally : process kill. Actually sends a kill signal to the selected process(es).

pkill has a fake twin brother : pgrep : one selects, the other kills .
They share the process selection options, so the workflow is usually :

  1. tune pgrep options to match the right process(es)
  2. use the same selection options to "pkill" them
For this reason, options related to process selection are detailed in the pgrep article only.

Flags

Flag Usage
-ssignal
--signal signal
  • send the specified signal to the selected process(es)
  • signal can either be the numeric or symbolic value

Example

example
mail

passwd

Usage

change user password

Flags

Flag Usage
-d --delete delete a user's password (make it empty)
-i days
--inactive days
This option is used to disable an account after the password has been expired for a number of days. After a user account has had an expired password for days days, the user may no longer sign on to the account.
-l --lock lock the password of the named account by changing the encrypted password into an invalid string in /etc/password. This is done by prefixing the encrypted string with an !.
passwd --lock does not actually disable the account : details.
-S --status Display account status information, consisting of 7 fields (see human-friendly equivalent : chage -l stuart) :
  1. user's login name
  2. indicates if the user account has :
    • L : a locked password (details : 1, 2)
    • NP : no password
    • P : a usable password
  3. date of the last password change
  4. password minimum age, in days (see chage)
  5. password maximum age, in days
  6. password warning period : see -w
  7. password inactivity period : see -i
-u --unlock unlock the password of the named account. This option re-enables a password by removing the leading ! added by -l.
-w days
--warndays days
Set the number of days of warning before a password change is required. days is the number of days prior to the password expiring that a user will be warned that his/her password is about to expire.

Example

Let's try this !

useradd bob; grep bob /etc/shadow
bob:!:17547:0:99999:7:::	new passwordless account is disabled
passwd bob; grep bob /etc/shadow
Enter new UNIX password: 123456
Retype new UNIX password: 123456
passwd: password updated successfully
bob:$6$OXeqopLJ$8xicqHptSgZ/JuUFFV1az4keocSKSa53bAC9BFdMRqn1d92ltyXI1Mc2Vx2a708Jhsvszeduuvm/49/uPp6FP1:17547:0:99999:7:::	the encrypted password
passwd -l bob; grep bob /etc/shadow
passwd: password expiry information changed.
bob:!$6$OXeqopLJ$8xicqHptSgZ/JuUFFV1az4keocSKSa53bAC9BFdMRqn1d92ltyXI1Mc2Vx2a708Jhsvszeduuvm/49/uPp6FP1:17547:0:99999:7:::	the extra ! shows it's locked
passwd -u bob; grep bob /etc/shadow
passwd: password expiry information changed.
bob:$6$OXeqopLJ$8xicqHptSgZ/JuUFFV1az4keocSKSa53bAC9BFdMRqn1d92ltyXI1Mc2Vx2a708Jhsvszeduuvm/49/uPp6FP1:17547:0:99999:7:::	unlocked
passwd -d bob; grep bob /etc/shadow
passwd: password expiry information changed.
bob::17547:0:99999:7:::	empty password
passwd bob; grep bob /etc/shadow
Enter new UNIX password: 123456
Retype new UNIX password: 123456
passwd: password updated successfully
bob:$6$nAmbe8KT$Dl3GXxWEjhRhuCPUA7dFyA7MZvAXtkAVJm14n0asq9OVioh1b1OK2mE0pYmTJqKtnY.kz5jGiaEJhs1JAEkTi.:17547:0:99999:7:::	password is back. Same clear text, different crypted text
passwd -dl bob; grep bob /etc/shadow
passwd: password expiry information changed.
bob:!:17547:0:99999:7:::	now empty AND locked
passwd -u bob; grep bob /etc/shadow
passwd: unlocking the password would result in a passwordless account.
You should set a password with usermod -p to unlock the password of this account.
bob:!:17547:0:99999:7:::	no change : will not make a passwordless account
passwd bob; grep bob /etc/shadow
Enter new UNIX password: 123456
Retype new UNIX password: 123456
passwd: password updated successfully
bob:$6$fyixHXv.$YT56pM8ExEtfsght7E8cxyP1EYmLwrBp1mW8sJ/ZVn8J.K0G9gQUC7mMycqXM0lAda1/.3KU.4Q56CtT8OtcH/:17547:0:99999:7:::	password added
usermod --expiredate 1 bob; grep bob /etc/shadow
bob:$6$fyixHXv.$YT56pM8ExEtfsght7E8cxyP1EYmLwrBp1mW8sJ/ZVn8J.K0G9gQUC7mMycqXM0lAda1/.3KU.4Q56CtT8OtcH/:17547:0:99999:7::1:
userdel -r bob; grep bob /etc/shadow
(nothing)
mail

parted

Usage

A partition manipulation program

Example

Interactive parted session

  • parted
  • (parted) select /dev/sdb
    Utilisation de /dev/sdb
  • (parted) mkpart primary ext2 0 -0
    Avertissement: L'alignement de la partition ainsi définie n'est pas optimal au niveau performance.
    Ignorer/Ignore/Annuler/Cancel?
    Alignment can NOT be optimal since we're using the whole disk.
    Ignore
    (nothing)
  • (parted) print
    Modèle: VMware Virtual disk (scsi)
    Disque /dev/sdb : 21,5GB
    Taille des secteurs (logiques/physiques): 512B/512B
    Table de partitions : msdos
    Disk Flags:
    
    Numéro	Début	Fin	Taille	Type	Système de fichiers	Fanions
     1	1049kB	21,5GB	21,5GB	primary
  • (parted) rm 1
    (nothing)
  • (parted) print
    Modèle: VMware Virtual disk (scsi)
    Disque /dev/sdb : 21,5GB
    Taille des secteurs (logiques/physiques): 512B/512B
    Table de partitions : msdos
    Disk Flags:
    
    Numéro	Début	Fin	Taille	Type	Système de fichiers	Fanions
  • (parted) quit
  • Unambiguous abbreviations are allowed : p for print, or u for unit
  • Commands are case-insensitive
(source, index of commands)

Non-interactive parted session

Create a partition using the whole disk

  • parted -a optimal -s /dev/sdb -- mkpart primary ext2 0 -1s
  • parted -a optimal -s /dev/sdb -- mkpart primary ext2 0 -0
  • ext2 is there just to set the partition flag. No filesystem will be created.
  • Negative numbers count back from the end of the disk
  • If no unit is given, the default unit is assumed defaults to compact. Read about unit for details
  • -1s means "the last sector of the disk"

Check

  • parted -l displays everything
  • parted /dev/sdb print

Delete a partition

parted /dev/sdb rm 1

When parted complains Error: Can not open /dev/sdc - unrecognized disk label.

Well, it's all in the error message : /dev/sdc has a disk label parted is unable to handle.
In my (possibly specific) case, I was trying to manage virtual disks on virtual machines with VMware. And these disks simply had no label. So let's add one prior to any operation :

parted -a optimal -s /dev/sdc -- mklabel msdos mkpart primary ext2 0 -1s

mail

parallel

Usage

parallel (GNU Parallel) is not a standard shell command, but can be installed on Debian with the parallel package.

parallel is designed to build and execute shell command lines from standard input in parallel and should be straightforward for xargs and tee enthusiasts. parallel can often be used as a substitute for xargs or cat something | bash command.

synopsis :

  • parallel [options] [command [arguments]] < list_of_arguments
  • parallel [options] [command [arguments]] ( ::: arguments | :::+ arguments | :::: argfile(s) | ::::+ argfile(s) ) ...
The parallel command -- arguments syntax has been replaced by parallel command ::: arguments.

Flags

Flag Usage
-j maxJobs run maxJobs jobs simultaneously
-i Enable argument placeholder (see examples below)
-n nbArgs number of arguments to pass to a command at a time. Defaults to 1.

Example

Parallel downloads with wget.
parallel echo Hello world ::: 1 2 3
Hello world 1
Hello world 2
Hello world 3
parallel echo Hello world ::: {1..10}
Hello world 1
Hello world 2
...
Hello world 10
tmpFile=$(mktemp XXX.tmp); parallel "echo hi >> $tmpFile; sleep 1; echo bye >> $tmpFile" ::: 1 2 3; cat "$tmpFile"; rm "$tmpFile"
hi
hi
hi
bye 1
bye 2
bye 3
tmpFile=$(mktemp XXX.tmp); parallel "echo hi_{} >> $tmpFile; sleep 1; echo bye >> $tmpFile" ::: 1 2 3; cat "$tmpFile"; rm "$tmpFile"
hi_1
hi_2
hi_3
bye
bye
bye
tmpFile1=$(mktemp XXX.tmp); tmpFile2=$(mktemp XXX.tmp); seq 1 5 >>"$tmpFile1"; parallel -j 3 "echo \"Start {}\">>$tmpFile2; sleep 1; echo \"Stop {}\">>$tmpFile2" < "$tmpFile1"; cat "$tmpFile2"; rm "$tmpFile1" "$tmpFile2"
Start 1
Start 2
Start 3
Stop  1
Stop  2
Stop  3
Start 4
Start 5
Stop  4
Stop  5
listOfItemsToProcess=$(mktemp XXX.tmp); nbParallelProcesses=3; outputHistory=$(mktemp XXX.tmp); seq 1 5 >>"$listOfItemsToProcess"; parallel -j $nbParallelProcesses "echo \"START {}\">>$outputHistory; sleep 1; echo \"STOP {}\">>$outputHistory" < "$listOfItemsToProcess"; cat "$outputHistory"; rm "$listOfItemsToProcess" "$outputHistory"
START 1
START 2
START 3
STOP 1
STOP 2
STOP 3
START 4
START 5
STOP 4
STOP 5
mail

pgrep

Usage

process grep : look through the currently running processes and lists the process IDs which match the selection criteria to stdout. All the criteria have to match.

pgrep options pattern

pgrep foo will match all processes whose executable file name (i.e. process name) matches foo. If foo is part of the path but not of the executable file name, pgrep will find no match. To match against the executable file name and path, consider -f.

pattern (source) :

is an ERE for matching against
  • the process name (i.e. binary name, default behavior)
  • the whole command line, with -f

Flags

Flag Usage
-f --full Match pattern against the full command line rather than just against the process name (binary)
-l --list-name Return the PID + process name
-n --newest Select only the newest (most recently started) of the matching processes
-r processState
--runstates processState
  • match processes having the processState state
  • processState is one of D, R, S, Z,
-u UID -u bob Match processes owned by UID / Bob (effective UID)
-U UID -U bob Match processes owned by UID / Bob (real UID)
-v Negates the matching

Example

select + kill a process :

Let's imagine there is a
tail -f path/to/someFile
process running somewhere :
  • get its PID :
    pgrep -f "tail.*someFile"
  • kill it :
    pkill -f "tail.*someFile"
mail

ping

Flags

Flag Usage
(none) ping until stopped by CTRL-c
-b broadcast ping : ping -b x.x.x.255
-c n ping count : send n probe packets
-f ping flood : the local machine sends ping requests without waiting for an echo before sending the next request. For each sent request, a dot . is displayed, for each received echo, a backspace is displayed.
-i n ping with interval : wait n seconds between 2 packets. (By default, 1 second)
-I sourceInterface ping from Interface : sourceInterface can be either an interface name (eth0) or its IP address
-v verbose mode
This mode outputs the following line twice :
ping: socket: Permission denied, attempting raw socket...
which is more a debug message than the sign something is going wrong. You can safely ignore it (details).
-R Record Route
-s packetSize data-loaded ping : packetSize bytes (+ headers) are sent
-w deadline timeout, in seconds, before ping exits regardless of how many packets have been sent or received
-W timeout time to wait for a response, in seconds

Example

Flooding from a FreeBSD host

ping -f -D -s 1472 x.x.x.x
sends a flood (-f) unfragmented (-D), data-loaded (-s) ping. The 1472 payload bytes are the maximum since we're sending an unfragmented packet on a LAN, where the MTU is 1500 bytes.
  • 1472 (payload) + 8 (IP headers) + 20 (ICMP ECHO REQUEST) = 1500 bytes
  • When receiving such input, an XP host has its network about 75% busy, and CPU about 30-40% busy.

By default, ICMP_ECHO_REPLY is disabled in XP SP1.

Destination host unreachable

You'll often get the Destination host unreachable error message with ping. This actually means you are 1 step ahead of "no response at all", which is good news but does not help much either . To further investigate, consider traceroute.
(see also)
mail

paste

Usage

Concatenate the corresponding lines of the input files.

Example

Considering 2 files :
  • file1
    file1, line1
    file1, line2
    file1, line3
  • file2
    file2, line1
    file2, line2
    file2, line3
paste file1 file2
file1, line1	file2,line1
file1, line2	file2, line2
file1, line3	file2, line3
mail

pwgen

Installed with the Debian package

pwgen

Usage

pwgen generates passwords which are designed to be easily memorized by humans, while being as secure as possible. Human-memorable passwords are never going to be as secure as completely random passwords. In particular, passwords generated by pwgen without the -s option should not be used in places where the password could be attacked via an off-line brute-force attack.

Flags

Flag Enabled by default Usage
-0 --no-numerals no number in the generated password
-1 prints the generated passwords one per line
-A --no-capitalize no capital letter in the generated password
-c --capitalize yes include at least one capital letter in the password
-N number --num-passwords=number generate number passwords instead of generating a screenful
-n --numerals yes include at least one number in the password
-s --secure generate completely random, hard-to-memorize passwords
-y --symbols include at least one special character in the password

Example

Generate a screenful of passwords :

Basic passwords : alphanumeric n-characters long ([a-zA-Z0-9]{n}) :

pwgen n

Truly secure passwords : strong but hard to memorize :

Some rules for password strength and safety of what's behind this password :
  • the size of the character set matters more than the password length :
    character set length combinations vs length of password
    [0-9] 10
    • 3 characters : 1000
    • 4 characters : 10000
    [a-z] 26
    • 3 characters : 263 = 17576
    • 4 characters : 264 = 456976
    Every character added to the password length multiplies the number of combinations by "length of the character set".
  • one of the most important (and forgotten) thing regarding password security is changing it regularly : you're still safe if you've changed your password before it's cracked. In other words : if it takes 2 years to brute-force your webmail password, and if it's stolen at anytime, you're still ok if you change it every year.
  • these should only be used for machine passwords, since otherwise it's almost guaranteed that users will simply write the password on a piece of paper taped to the monitor...
pwgen -ys 16

Intermediate passwords : stronger than basic ones but still possible to memorize + have distinct passwords for all accounts + change them regularly :

pwgen -y 14

Generate some funny words :

This is for fun only : the words generated with the methods below —albeit created with a password generator— must NOT be used as passwords. Indeed, the syntax rules (lowercase characters only, patterns of alternating consonants / vowels) makes them extremely poor passwords.

These methods can be used to make up characters / places / cities / ... names in fictional writing work, for instance (don't forget to adapt the "selection RegExp" to the length of words you wish to generate).

single word :
wordLength=10; generateOneWord() { pwgen -0 -1 -A "$wordLength" | grep -Ei '^[b-df-hj-np-tv-xz][aeiouy]{1,2}[b-df-hj-np-tv-xz][aeiouy]{1,2}[b-df-hj-np-tv-xz]{1,2}'; } ; until generateOneWord ; do true; done
multiple words :
wordLength=10; nbWordsToGenerate=12; nbGeneratedWords=0; generateOneWord() { consonant='[b-df-hj-np-tv-xz]'; vowel='[aeiouy]'; pwgen -0 -1 -A "$wordLength" | grep -Ei "^($consonant$vowel{1,2}$consonant$vowel{1,2}$consonant{1,2}$vowel{2}|$vowel$consonant{1,2}$vowel{1,2}$consonant{1,2}$vowel{1,2}$consonant)"; } ; until [ "$nbGeneratedWords" == "$nbWordsToGenerate" ] ; do generateOneWord && nbGeneratedWords=$((nbGeneratedWords+1)); done