Bash Index : C - The 'C' Bash commands : description, flags and examples

1. create a user account :
   adduser bob
2. get account initial information :
   grep bob /etc/{passwd,shadow}

   /etc/passwd:bob:x:1001:1001:,,,:/home/bob:/bin/bash			a password is set
   /etc/shadow:bob:$6$f9YcHiE8Fc5Ho4Z4bALVuc/:18535:0:99999:7:::	the encrypted password

   passwd -S bob

   bob P 09/30/2020 0 99999 7 -1

   chage -l bob

   Last password change                                    : Sep 30, 2020
   Password expires                                        : never
   Password inactive                                       : never
   Account expires                                         : never
   Minimum number of days between password change          : 0
   Maximum number of days between password change          : 99999
   Number of days of warning before password expires       : 7

3. lock the password with passwd :
   passwd -l bob

   passwd: password expiry information changed.

   grep bob /etc/{passwd,shadow}

   /etc/passwd:bob:x:1001:1001:,,,:/home/bob:/bin/bash
   /etc/shadow:bob:!$6$f9YcHiE8Fc5Hxdvk$YW1TwjbGerIKvp8vY.rRPBESiIduTOI8.eDzTWjQZyhPMTd6SMl2Ju21dDvEJLTFjeru2YEaWa7o4Z4bALVuc/:18535:0:99999:7:::

   passwd -S bob

   bob L 09/30/2020 0 99999 7 -1

   chage -l bob

   (same as above)

4. lock the password with passwd :
   passwd -u bob

   passwd: password expiry information changed.

   grep bob /etc/{passwd,shadow}

   /etc/passwd:bob:x:1001:1001:,,,:/home/bob:/bin/bash
   /etc/shadow:bob:$6$f9YcHiE8Fc5Hxdvk$YW1TwjbGerIKvp8vY.rRPBESiIduTOI8.eDzTWjQZyhPMTd6SMl2Ju21dDvEJLTFjeru2YEaWa7o4Z4bALVuc/:18535:0:99999:7:::

   passwd -S bob

   bob P 09/30/2020 0 99999 7 -1

   chage -l bob

   (same as above)

5. expire the account with usermod :
   usermod -e 1 bob

   (returns nothing)

   passwd -S bob

   bob P 09/30/2020 0 99999 7 -1

   chage -l bob

   Last password change                                    : Sep 30, 2020
   Password expires                                        : never
   Password inactive                                       : never
   Account expires                                         : Jan 02, 1970
   Minimum number of days between password change          : 0
   Maximum number of days between password change          : 99999
   Number of days of warning before password expires       : 7

6. un-expire the account with usermod :
   usermod -e '' bob

   (returns nothing)

   passwd -S bob

   bob P 09/30/2020 0 99999 7 -1

   chage -l bob

   Last password change                                    : Sep 30, 2020
   Password expires                                        : never
   Password inactive                                       : never
   Account expires                                         : never
   Minimum number of days between password change          : 0
   Maximum number of days between password change          : 99999
   Number of days of warning before password expires       : 7

7. expire the account with chage :
   chage -E 1 bob

   (returns nothing)

   passwd -S bob

   bob P 09/30/2020 0 99999 7 -1

   chage -l bob

   Last password change                                    : Sep 30, 2020
   Password expires                                        : never
   Password inactive                                       : never
   Account expires                                         : Jan 02, 1970
   Minimum number of days between password change          : 0
   Maximum number of days between password change          : 99999
   Number of days of warning before password expires       : 7

8. un-expire the account with chage :
   chage -E -1 bob

   (returns nothing)

   passwd -S bob

   bob P 09/30/2020 0 99999 7 -1

   chage -l bob

   Last password change                                    : Sep 30, 2020
   Password expires                                        : never
   Password inactive                                       : never
   Account expires                                         : never
   Minimum number of days between password change          : 0
   Maximum number of days between password change          : 99999
   Number of days of warning before password expires       : 7

9. delete the user account :
   userdel -r bob

Changing the aging information for stuart
Enter the new value, or press ENTER for the default

	Minimum Password Age [0]: 
	Maximum Password Age [99999]: 
	Last Password Change (YYYY-MM-DD) [2018-06-08]: 
	Password Expiration Warning [7]: 
	Password Inactive [-1]: 
	Account Expiration Date (YYYY-MM-DD) [1970-01-02]: -1

tmpFile1=$(mktemp); tmpFile2=$(mktemp); echo 'FILE1' > "$tmpFile1"; echo 'FILE2' > "$tmpFile2"; for i in {1..3}; do echo "line $i" >> "$tmpFile1"; done; for i in {3..5}; do echo "line $i" >> "$tmpFile2"; done; cat "$tmpFile1" "$tmpFile2"; for mode in 1 2 3 12; do echo -e "\n'-$mode' mode :"; comm -$mode "$tmpFile1" "$tmpFile2"; done; rm "$tmpFile1" "$tmpFile2"

FILE1			result of cat file1 file2
line 1
line 2
line 3
FILE2
line 3
line 4
line 5

'-1' mode :
FILE2
	line 3		this line is common to both files, others are unique to file2
line 4
line 5

'-2' mode :
FILE1
line 1
line 2
	line 3		this line is common to both files, others are unique to file1

'-3' mode :		common lines are hidden
FILE1
	FILE2
line 1
line 2
	line 4
	line 5

'-12' mode :		displaying only common lines
line 3

• root
• the file owner
• a process with the CAP_FOWNER capability

drwx------ 2 bob developers 4096 Mar 23 16:35 dir
-rw------- 1 bob developers    0 Mar 23 16:35 file

drwx--x--x 2 bob developers 4096 Mar 23 16:35 dir
-rw------- 1 bob developers    0 Mar 23 16:35 file

• with the octal syntax :
  chmod 644 myFile
• with the symbolic syntax :
  chmod u=rw,g=r,o=r myFile

cryptsetup luksFormat

set up a new dm-crypt device in LUKS encryption mode

cryptsetup luksOpen

open an existing dm-crypt device

cryptsetup luksClose

close a dm-crypt device

We ALL did it, but now, the cat | construct should ring the bell of uselessness and waste of resources (even though modern computers have so much CPUs and GB of RAM to waste that such commands will pass unsuspected ).

Actually, any extra | induces an additional process fork, which is an expensive CPU-time operation (especially on loops). Moreover, cat someFile | reads the whole file before piping it to the 2^nd command, which delays its execution for big files.

man cp says : if an existing destination file cannot be opened, remove it and try again (this option is ignored when the -n option is also used)
The final part (between parenthesis) is clear (-n disables -f), so let's concentrate on the beginning :

if an existing destination file cannot be opened, remove it and try again

man cp says : If a file descriptor for a destination file cannot be obtained, attempt to unlink the destination file and proceed. It also gives the detailed procedure :

• when sourceFile is a regular file
• and when destinationFile exists
• if a file descriptor for destinationFile can be obtained via actions such as open() (i.e. if destinationFile is writable by the current user)
  • THEN copy sourceFile into destinationFile
  • OTHERWISE delete destinationFile (if enough privileges to do so), then copy sourceFile as destinationFile

Round 1		file2 didn't exist already, so that was easy !

Round 2		Piece of cake !!!

Round 3		w bit is set, no need to force

Round 4a
cp: cannot create regular file 'file2': Permission denied

Round 4b	w bit is NOT set, -f required

Round 5a
cp: cannot create regular file 'file2': Permission denied

Round 5b	no permission bit is set, -f required

Now if I try with file2 being a file I don't own, and on which I only have read permission (r--) :

cp: cannot create regular file 'file2': Permission denied
cp: cannot remove 'file2': Operation not permitted

As a conclusion, the -f / --force cp flag only serves to overwrite my own files if I have denied myself write access on them.

Make a backup of an existing destination :

touch myFile; cp myFile myFile_COPY; cp -b myFile myFile_COPY; ls -1; rm myFile*

myFile
myFile_COPY
myFile_COPY~

The default options do not overwrite the backup :

echo initial > myFile; cp myFile myFile_COPY; cp -b myFile myFile_COPY; echo modified > myFile; cp -b myFile myFile_COPY; ls -1; cat myFile myFile_COPY myFile_COPY~; rm myFile*

myFile
myFile_COPY
myFile_COPY~
modified
modified
initial		contents of myFile_COPY~

Make numbered backups :

echo 'initial version' > myFile; cp --backup=numbered myFile myFile_COPY; for i in {1..5}; do echo "edit $i" > myFile; cp --backup=numbered myFile myFile_COPY; done; for i in myFile*; do echo -en "$i :|"; cat "$i"; done | column -s '|' -t; rm myFile*

myFile :           edit 5
myFile_COPY :      edit 5
myFile_COPY.~1~ :  initial version
myFile_COPY.~2~ :  edit 1
myFile_COPY.~3~ :  edit 2
myFile_COPY.~4~ :  edit 3
myFile_COPY.~5~ :  edit 4

Make suffixed backups :

touch myFile; cp myFile myFile_COPY; cp -b --suffix='_OLD' myFile myFile_COPY; ls -1; rm myFile*

myFile
myFile_COPY
myFile_COPY_OLD

#!/usr/bin/env bash

for i in {1..2}; do
	echo $i
	continue
	echo 'this will never be executed'
done

1
2

#!/usr/bin/env bash

for i in {1..2}; do
	for j in {a..b}; do
		echo $i$j
		continue
		echo 'this will never be executed'
	done
done

1a
1b
2a
2b

#!/usr/bin/env bash

for i in {1..2}; do
	for j in {a..b}; do
		echo $i$j
		continue 2
		echo 'this will never be executed'
	done
done

1a
2a

for i in {1..3}; do echo col1_$i col2_$i; done | column -c 40

col1_1 col2_1	col1_3 col2_3
col1_2 col2_2

for i in {1..3}; do echo col1_$i col2_$i; done | column -t

col1_1	col2_1
col1_2	col2_2
col1_3	col2_3

for i in {1..3}; do echo col1_$i col2_$i; done | column -s 'l' -t

co	1_1 co	2_1
co	1_2 co	2_2
co	1_3 co	2_3

crontab is used to :

• submit jobs definition files to cron
• list the contents of a cron table (personal or different user)

Unless explicitly specified with -u, all crontab commands apply to the current user.

crontab works so that you cannot append a new job at the end of a table. Instead, you have to :

1. extract the whole content of the table into a file :
   • for the current user : crontab -l > /tmp/crontab
   • as root, for the user bob : crontab -l -u bob > /tmp/crontab.bob
   You can also launch edition with vi (default) or your favorite editor at the same time with the -e option. If no crontab is available, just create a new text file and go to the next step.
2. edit the file to add / modify / delete jobs with your favorite text editor. The file MUST end with a blank line.
   • you may define variables (aka environment settings) in the key = value form (spaces around the = sign are allowed) but there are limitations : these are not supported :
     • environmental substitutions : somePath = $HOME/bin
     • replacement of variables :

       A=1
       B=2
       C=$A $B

     • tilde expansion : PATH = ~/bin
   • about $PATH :
     • it is inherited from the environment (source : man -P 'less -p "PATH is inherited"' 5 crontab)
     • should you need to specify any non-standard value :
       • uncomment + edit the line in the crontab file header : #PATH=/usr/local/sbin:/usr/local/bin:
       • it is affected by the limitations above
3. register the new crontab :
   • for the current user : crontab /tmp/crontab
   • as root, for the user bob : crontab -u bob /tmp/crontab.bob
   This overwrites the previous crontab.

fields :

wildcards :

aliases (source) :

• /etc/cron.d/*
• /etc/cron.daily
• /etc/cron.hourly
• /etc/cron.monthly
• /etc/cron.weekly
• /etc/crontab
• /etc/default/cron
• /var/spool/cron/crontabs/kevin (Debianoids only)

With the exception of certain errors in the time fields, errors are not reported until CRON runs the command. All error messages and output are mailed to the crontab owner (or to any address specified with MAILTO=bob@example.com), unless they are silenced by redirecting stdout and stderr to /dev/null with >/dev/null 2>&1.

Output is mailed to the user because there is no real terminal on which the cronjobs are being executed. Therefore, there is no screen to display the errors. Also, there is no keyboard to accept input. Does that mean you cannot give input to a CRON job? No. Think back to the discussion on shell scripts. We can redefine stdin, stdout and stderr. This way they can all point to files and behave as we expect.

Keep in mind that CRON is not exact. It synchronizes itself to the top of each minute. On a busy system in which you lose clock ticks, jobs may not be executed until a couple minutes after the scheduled time. In addition, there may be other processes with higher priorities that delay CRON jobs. In some cases, (particularly on very busy systems) jobs might end up being skipped if they are run every minute.

• /var/log/cron
• /var/log/cronexecutionDate

• for user in $(cut -d : -f1 /etc/passwd); do echo $user"'s CRONTAB :"; crontab -l -u $user; done | less
• or, as root : less /var/spool/cron/crontabs/*

  /var/spool/cron/crontabs/* files are not intended to be edited directly. Use the crontab utility to edit CRON jobs.

Let's say we wanted to run a command not at 10 minutes after the hour, but every ten minutes. We could make an entry that looked like this :

0,10,20,30,40,50	*	1,16	*	1-5	/usr/local/bin/command

This runs every 10 minutes: at the top of the hour, 10 minutes after, 20 minutes after, and so on. To make life easier, we could simply create the entry like this :

*/10	*	1,16	*	1-5	/usr/local/bin/command

The /n says that within the specific interval (in this case, every minute), run the command every n minutes; in this case, every 10 minutes.

We can also use this even when we specify a range. For example, if the job was only supposed to run between 20 minutes after the hour and 40 minutes after the hour, the entry might look like this :

20-40	*	1,16	*	1-5	/usr/local/bin/command

What if you wanted it to run at these times, but only every three minutes? The line might look like this :

20-40/3	*	1,16	*	1-5	/usr/local/bin/command

To make things even more complicated, you could say that you wanted the command to run every two minutes between the hour and 20 minutes after, every three minutes between 20 and 40 minutes after, then every 5 minutes between 40 minutes after and the hour :

0-20/2,21-40/3,41-59/5	*	1,16	*	1-5	/usr/local/bin/command

To make the fun last longer, let's imagine you have 2 jobs you want to execute every 2 minutes (source) :

• jobEven on even minutes : 2, 4, 6, ..., 58, 00
• jobOdd on odd minutes : 1, 3, 5, ..., 59

*/2	*	*	*	*	jobEven
1-59/2	*	*	*	*	jobOdd

And what if I want a job to run every n minutes with jobExecutionMinute % n != 0 ? (source)

1,6,11,16,21,26,31,36,41,46,51,56 * * * * /my/script
3,13,23,33,43,53 * * * * /my/other/script

1-56/5 * * * * /my/script
3-53/10 * * * * /my/other/script

# 1st monday of month at 6h30
30 6 * * 1 [ $(/bin/date +\%d) -lt 8 ] && /sbin/init 6

# Last wednesday of every month, at 6h30
#30 6 * * 3 [[ $(date +\%d) == $(echo "$(echo "$(cal -s )"|awk '{print $4}')"|tail -1) ]] && /sbin/init 6

# not tested yet
			

• chkconfig --list
• chkconfig --list httpd

These commands enable / disable daemons, but they don't start / stop them.

• chkconfig --level 345 nscd off
• chkconfig --level 3 httpd on

If no runlevel is specified, then all the possible runlevels are impacted :

chkconfig iptables off

(nothing)

chkconfig --list iptables

iptables	0:off	1:off	2:off	3:off	4:off	5:off	6:off

chkconfig iptables on

(nothing)

chkconfig --list iptables

iptables	0:off	1:off	2:on	3:on	4:on	5:on	6:off

1
0

• when listing services : chkconfig --list 2>&1 | less
• when taking actions on serviceName : chkconfig --del serviceName

In my case, this was caused by a broken link :

/etc/init.d/serviceName -> /installDir/serviceName/bin/scripts/serviceName.sh

1. add 2 commented lines in the header of scriptName :

   # chkconfig: runlevels startPriority stopPriority
   # description: description

   • runlevels : list of runlevels on which this script should start by default. To mean "none", just indicate -.
   • startPriority : (explicit)
   • stopPriority : (explicit)
   • description : (explicit) This can span on multiple lines with a trailing \ on each continued line.
2. copy the script as /etc/init.d/scriptName
3. register this new script : chkconfig --add scriptName
   This will create the appropriate symlinks :
   • /etc/rc.d/rcrunlevel.d/KstopPriorityscriptName -> ../init.d/scriptName
   • /etc/rc.d/rcrunlevel.d/SstartPriorityscriptName -> ../init.d/scriptName

• n : n^th, counted from 1
• n- : from n^th to end of line
• n-m : from n^th to m^th (included)
• -m : from first to m^th (included)
• x,y,z : the x^th, the y^th and the z^th
  the data fields will be displayed following the order they have in the input, not the order you list them

echo 'aze rty uio' | cut -d ' ' -f 2,3

rty uio

echo a,b,c | cut -d ',' -f2,3

b,c

echo a,b,c | cut -d ',' -f2,3 --output-delimiter=' '

b c

for i in {1..26}; do echo {a..z} | cut -d ' ' -f 1-$i; done

a
a b
a b c

a b c d e f g h i j k l m n o p q r s t u v w x y z

echo hello | cut -c 2

e

echo hello | cut -c -2

he

echo hello | cut -c 2-

llo