Bash Index : D - The 'D' Bash commands : description, flags and examples

dirname

strip last component from file name, i.e. returns the name of the directory containing the file passed as parameter.

disown

Usage :

To do so :

source : help disown

deborphan

Usage :

deborphan is not a Bash command, but a Debian-specific utility. It can be installed via the deborphan package.

deborphan looks for orphaned packages, that is packages which are not required by any other package upon your system.

Flags :

Flag Usage
(none) only list packages which are in the libraries section of the Debian archive, as these are the most likely candidates for removal
-s --show-section show the sections the packages are in
-z --show-size show the installed size in KiB of the listed packages
--guess-data show library (default) + data orphaned packages
--guess-all show all orphaned packages
There are many --guess-* commands available. See the GUESSING section of man deborphan.

Example :

List orphaned packages :

deborphan
libssl1.0.0
libgnome2-bin
liblognorm1
libpsl0
libprocps3
libboost-iostreams1.55.0
deborphan --guess-all -sz
	111 main/net			libxtables10
	183 main/python			python3-cffi-backend
	352 main/java			libservlet2.5-java
	 53 main/perl			libclass-isa-perl
	458 main/admin			software-properties-common
This doesn't require root privileges since it just reads the installed packages lists.

Remove orphaned packages :

As root :

deborphan | xargs apt-get purge

Commands below have the -y flag which will uninstall packages with no prompt (but there should be no risk as apt-get is smart enough to avoid shooting its own foot ).

dmesg

Usage :

dmesg's main usage is to display messages from the kernel ring buffer. It lists events in the form : 

[timestamp] message
where timestamp is the number of seconds.nanoseconds since the system booted.

Information reported by dmesg is also available in /var/log/kern.log

The kernel ring buffer :

The kernel ring buffer is a memory area containing kernel messages. It's a fixed-size FIFO area.

Flags :

Flag Usage
-T --ctime Print human-readable timestamps

Be aware that the timestamp could be inaccurate!

declare

Usage :

declare (or its synonym typeset) is used to change properties of variables

Flags :

Flag Usage
-a myVariable declare myVariable is an indexed array (example)
-A myVariable declare myVariable is an associative array (example)
-f list existing Bash functions (names + code)
-F list existing Bash functions (names only)
-i myVariable declare myVariable is an integer
-r myVariable declare myVariable is read-only
Both are valid :
  • declare + assign at once : 
    declare -... myVariable=value
  • declare then assign : 
    declare -... myVariable
myVariable=value

Example :

Declare a variable as readonly :

declare -r a=1; echo $a; a=2 
1
-bash: a: readonly variable
Or : readonly b=2; echo $b; b=1 
2
-bash: b: readonly variable

readonly variables exist until the end of the shell process.

How to unset a readonly variable ? (source)

readonly a=1; echo $a
1
unset a
bash: unset: a: cannot unset: readonly variable
cat << EOF | gdb
> attach $$
> call unbind_variable("a")
> detach
> EOF
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.

(gdb) Attaching to process 29720
Reading symbols from /bin/bash...(no debugging symbols found)...done.

Loaded symbols for /lib/x86_64-linux-gnu/libnss_files.so.2
0x00007fd61bed006e in waitpid () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) $1 = 0
(gdb) Detaching from program: /bin/bash, process 29720
(gdb) quit
echo $a
(nothing)

What's the difference between declare and readonly ? (source)

The variable scope :
  • declare makes it local
  • readonly makes it global

As of bash 4.2, declare -gr seems to be identical to readonly.

df

Usage :

Report disk free space

You may face weird situations where

Read more about this here.

Should you need to list several filesystems (but not all of them) as well as the header line (describing data fields, but language-dependent), there's no need to use grep, just remember df accepts a list of filesystems :
df -h /var/lib/mysql /var/lib/mysql/backup
Sys. fich.		Taille	Util.	Dispo	Uti%	Monté sur			Language-dependant header line
/dev/mapper/vg1-mysql	55G	15G	37G	29%	/var/lib/mysql
/dev/mapper/vg2-dump	20G	9,9G	8,9G	53%	/var/lib/mysql/backup

Flags :

Flag Usage
-h --human-readable print sizes in human readable format (e.g. 1K, 234M, 2G)
-i --inodes report inode usage instead of block usage
-P --portability use the POSIX output format :
  • no nice tabular display anymore
  • a single line per filesystem, even though the filesystem name is long
To be used in a df -P | grep context.
-T --print-type print filesystem type
This can be used to determine which filesystem a file belongs to :
df -T $HOME
Filesystem			Type	1K-blocks	Used	Available	Use%	Mounted on
/dev/mapper/caramba--vg-home	ext4	25066604	4086168	19684052	18%	/home

Example :

Check filesystems for disk full/almost full :

  • When above 90% of used disk space, there may be No space left on device to create a new BIG file there
  • There may be no inodes left, even though some (kilo|mega|giga|tera)bytes are available
Check this with :
echo 'Disk usage :'; df -Ph | grep -E '(9.|100)%'; echo 'Inodes :'; df -Phi | grep '100%'
Alternate syntax :
This allows specifying a threshold much more easily :
df -Ph | tr -d '%' | awk '$5 > 90 { print $0 }'

Find usage percent of filesystem holding /path/to/myFile :

df is actually smart enough to determine which filesystem holds /path/to/myFile and report the usage :

df -h "$HOME/.bashrc"
Filesystem			Size	Used	Avail	Use%	Mounted on
/dev/mapper/caramba--vg-home	24G	3.9G	19G	18%	/home
One-liner for scripts :
df -h "$HOME/.bashrc" | awk '$6 ~ "^/" { print $5 }'
18%

diff

Usage :

Compare 2 files : diff file1 file2
diff can also be used to compare 2 directories to list :
  • which files exist only once
  • differences between files that are found in both directories

Flags :

Flag Usage
-b --ignore-space-change ignore changes in the amount of white space
-B --ignore-blank-lines ignore changes where lines are all blank
-i --ignore-case ignore case in file content (don't mix this option up with --ignore-file-name-case)
-q --brief list only name of files having differences
-r compare files recursively when file1 and file2 are directories.
--suppress-common-lines Do not output common lines.
-u n -U n output n lines of unified context (n defaults to 3)
-w --ignore-all-space ignore all white space
-W n --witdh=n Display at most n print columns (defaults to 130, see tput cols)
-x pattern --exclude=pattern ignore files matching pattern
-y --side-by-side output in 2 columns
-Z --ignore-trailing-space ignore white space at line end

Example :

Compare 2 files side by side :

  • diff --color=always -Bbiy -W $(tput cols) file1 file2 | less -R
  • fileA='inventory_CW_STG_PCK.ini'
fileB='inventory_FE_BCR.ini'
diff --color=always --suppress-common-lines -Bbiy -W $(tput cols) "$fileA" "$fileB" | less -R
    All lines output with this command syntax have differences, even if they are not color-highlighted.

dig

Usage :

dig : DNS lookup utility
Forward DNS lookup (name into IP) :
  • dig name
  • dig name recordType
  • dig @dnsServer name
Reverse DNS lookup (IP to name) :
dig -x ip.add.re.ss

Example :

Investigate DNS lookup time :

#!/usr/bin/env bash

logFile=$0.csv
hostName='www.example.com'
serversList='DNS servers list'
echo "DNS Server;Time [ms]" > $logFile
for dnsServer in $serversList; do
	echo -n .
	echo -n "$dnsServer;" >> $logFile
	dig @$dnsServer $hostName | grep 'Query time' | cut -d ' ' -f 4 >> $logFile
done

Read TXT records of a domain :

dig google.com txt

Get a short response (IP only) :

Instead of dig www.example.com, which is rather verbose, get only the IP address using the +short flag : dig +short www.example.com

dd

Usage :

dd stands for data duplicator (source) but there is no consensus on the meaning of dd...

Flags :

Flag Usage
if=file input file. This can be :
of=file output file
bs=n[unit] block size
  • n : n bytes
  • nkB : n*1000 bytes
  • nK : n*1024 bytes
  • nMB : n*1000*1000 bytes
  • nM : n*1024*1024 bytes
  • nGB : n*1000*1000*1000 bytes
  • nG : n*1024*1024*1024 bytes
count=n generate n blocks of the specified size
seek offset
status=level the level of information to print to stderr :
  • none : suppresses everything but error messages
  • noxfer : suppresses the final transfer statistics
  • progress : shows periodic transfer statistics

Example :

How to create a test file having a specific size ?

For 100MB of 0s :
dd if=/dev/zero bs=1M count=100 of=some/file
There is a faster method (limited to ext4, btrfs, xfs and ocfs2) : fallocate. This command actually preallocates blocks to a file but doesn't write the whole file, which will look empty.
Suffices like k, m, g and t stand for KiB, MiB, GiB and TiB respectively.
fallocate -l 100m some/file

dpkg

Flags :

Flag Usage
-b myPackage --build myPackage build the myPackage.deb package (details)
-c packageName --contents packageName show contents of packageName
packageName has to be available on the local filesystem
-i packageName install the package packageName
packageName has to be available on the local filesystem
-I packageName --info packageName show information about packageName
packageName has to be available on the local filesystem
-l list all the installed packages
-L packageName List files installed by the package packageName
-P packageName Purge the package packageName : completely uninstall the package including binaries and configuration files
-r packageName remove the package packageName, leaving the configuration files untouched.
-s packageName --status packageName display status information about the package packageName. (this actually reads the package control file)
list packages on which packageName depends
-S someFile --search someFile display which package installed someFile. Check it :
directory='/usr/bin'; nbFiles=$(ls -1 "$directory" | wc -l); randomNumber=$(shuf -i 1-$nbFiles -n 1); randomFile=$(ls -1 "$directory" | sed -n "$randomNumber p"); echo "Random file : '$directory/$randomFile'"; dpkgSResult=$(dpkg -S "$directory/$randomFile"); echo "... is from package : '$dpkgSResult'"; packageName=$(echo "$dpkgSResult" | cut -d':' -f1); echo "... which also installed : "; dpkg -L "$packageName" | grep "$directory"
If this fails saying :dpkg-query: no path found matching pattern someFile, it may be because someFile is a symlink.

Exit Status :

dpkg -l status codes :

rc
r : the package was marked for removal
c : the configuration files are currently present in the system
See apt-get purge
ii
installed
List of all existing codes :

Example :

List dependencies of packageName (see also aptitude why) :

The packages packageName depends on :
  • dpkg -s packageName | grep "^Depends"
  • apt-cache show mysql-server | grep Depends
  • or even better : apt-cache depends mysql-server
Depends: mysql-server-5.5
The packages that depend on packageName (source) :
apt-cache rdepends mysql-server 
mysql-server
Reverse Depends:
  mysql-server-5.5
 |yubikey-val		meaning of this | ? Read below.
 |yubikey-ksm
 |spip
 |prometheus-mysqld-exporter
  openstack-cloud-services
What does the | in the output of apt-cache rdepends packageName mean ? (source) :
As seen above, apt-cache rdepends packageName lists the packages that depend on packageName :
apt-cache rdepends mutt
mutt
Reverse Depends:
  t-prot
  notmuch-mutt
  devscripts
  urlview
  urlscan

 |sylph-searcher	sylph-searcher depends on mutt or an alternate package (shown by the |)

  mixmaster
 |metche
  lbdb

  automysqlbackup	automysqlbackup depends on mutt, and there's no alternative
the listed packages have some kind of dependency on mutt (Depends, Recommends, Suggests, Enhances, ...)
check it : apt-cache show sylph-searcher metche automysqlbackup | grep -E '^Package|mutt'
Package: sylph-searcher
Suggests: sylpheed | claws-mail | mutt | wl	one of these will suffice

Package: metche
Depends: debconf (>= 0.5) | debconf-2.0, mutt | mailx | mail-transport-agent, bzip2, ucf	debconf AND (mutt OR mailx OR mail-transport-agent) AND bzip2 AND ucf

Package: automysqlbackup
Recommends: mutt	no alternative

Check whether packageName is installed :

  • dpkg -l | grep packageName
  • dpkg -l packageName &>/dev/null && echo 'INSTALLED' || echo 'NOT INSTALLED'

date

Usage :

Set, display and manipulate dates.

Flags :

Flag Usage
(none)
  • display the current date and time : Wed Aug 5 09:35:06 CEST 2015
  • you can specify the output format
-d dateString
--date dateString 		display time described by dateString, using keywords such as now, yesterday, today, tomorrow, day(s), week(s), month(s), year(s) :
  • date --date "now -60 days"
  • date --date=tomorrow
  • for unitOfTime in day week month year; do for number in {1..3}; do echo -en "+$number$unitOfTime :\t"; date --date "now +$number$unitOfTime"; done; done
more about date strings
-r file
--reference file 		display the last modification time of file
-s dateString
--set dateString 		set time described by dateString :
date --set="Oct 31 10:52:40"

Date strings (source) :

The --date='dateString' is a human-readable format such as next Thursday or 1 month ago. A date string may contain items indicating :

  • calendar date
  • time of day
  • time zone
  • day of the week
  • relative time
  • relative date
  • numbers

Keywords
  • now is a synonym of today. Check it : date --date now; date --date today
  • tomorrow is today + 24 hours. The same goes on with yesterday and today. Check it : date --date yesterday; date --date today; date --date tomorrow
Examples
The commands below were run on the Friday 19th of July.
the basics
date
Fri 19 Jul 2019 11:17:29 AM CEST
date --date="today"			same as above
Fri 19 Jul 2019 11:17:44 AM CEST
date --date="tomorrow"
Sat 20 Jul 2019 11:17:54 AM CEST
date --date="next monday"
Mon 22 Jul 2019 12:00:00 AM CEST
dates in the future / in the past
date --date="3 days"
Mon 22 Jul 2019 11:22:13 AM CEST
date --date="3 days ago"		ago ...
Tue 16 Jul 2019 11:22:19 AM CEST
date --date="-3 days"			and negative values compute dates in the past ...
Tue 16 Jul 2019 11:22:25 AM CEST
date --date="-3 days ago"		unless you combine both 
Mon 22 Jul 2019 11:37:53 AM CEST
date --date="-1 year month"
Sun 19 Aug 2018 12:32:02 PM CEST	+1 is implied for units without a number
(void) / this / next
date --date="thursday"
Thu 25 Jul 2019 12:00:00 AM CEST
date --date="this thursday"
Thu 25 Jul 2019 12:00:00 AM CEST
date --date="next thursday"
Thu 25 Jul 2019 12:00:00 AM CEST

date --date="friday"
Fri 19 Jul 2019 12:00:00 AM CEST
date --date="this friday"
Fri 19 Jul 2019 12:00:00 AM CEST
date --date="next friday"
Fri 26 Jul 2019 12:00:00 AM CEST

date --date="saturday"
Sat 20 Jul 2019 12:00:00 AM CEST
date --date="this saturday"
Sat 20 Jul 2019 12:00:00 AM CEST
date --date="next saturday"
Sat 20 Jul 2019 12:00:00 AM CEST
  • (void) implies this
  • this weekDay means :
    • if today is a weekDay : the current day
    • otherwise : next weekDay
future weekdays :
date --date="next monday"
Mon 22 Jul 2019 12:00:00 AM CEST
date --date="1 monday"			same as next
Mon 22 Jul 2019 12:00:00 AM CEST
date --date="2 monday"
Mon 29 Jul 2019 12:00:00 AM CEST
date --date="second monday"		second is "1 second", not "the second from now"
Mon 22 Jul 2019 12:00:01 AM CEST
date --date="third monday"
Mon 05 Aug 2019 12:00:00 AM CEST
date --date="fourth monday"
Mon 12 Aug 2019 12:00:00 AM CEST
date --date="tenth monday"
Mon 23 Sep 2019 12:00:00 AM CEST

Examples of output format :

Output Format string Usage
20170201 date +%Y%m%d compact representation of a date that can be used in a filename
20181116_152852 date +%Y%m%d_%H%M%S
2018-11-29 15:51:36 date '+%F %H:%M:%S' Quotes are necessary when the format string has a space, otherwise you'll get a date: extra operand ‘%H:%M:%S’ error message.
%F is a synonymous of %Y-%m-%d
2011-12-29_10-23-08 date +%F_%H-%M-%S (almost) same as above without the : characters, which is better if this timestamp is part of a filename
1014 date +%H%M at time format
jeu., 28 nov. 2013 15:10:46 GMT date +"%a, %d %b %Y %H:%M:%S GMT" If-Modified-Since HTTP header date format
1387202860 date +%s current Unix timestamp 
Wed Mar 15
Fri Sep 1
 date +"%a %b %-d" git log date format
the extra - prevents padding the day number with a leading 0 if <10.
this format ignores the year, which may have unexpected results if the Git history has commits on the "same day" of distinct years (which happened to me with Wed Dec 12 of 2018 and 2012)
 
Fri Mar 13 17:35:03 2020 +0100
 date +"%a %b %-d %H:%M:%S %Y %z" git log long date format (depending on personal settings / aliases /...)

Example :

Convert dates :

Unix timestamp into date (source) :
for unixTime in 1234567890 1500000000 1600000000 2000000000; do echo -e "UNIX time : $unixTime\t==>\tLOCAL TIME : $(date -d@$unixTime)"; done 
UNIX time : 1234567890	==>	LOCAL TIME : Sat Feb 14 00:31:30 CET 2009
UNIX time : 1500000000	==>	LOCAL TIME : Fri Jul 14 04:40:00 CEST 2017
UNIX time : 1600000000	==>	LOCAL TIME : Sun Sep 13 14:26:40 CEST 2020
UNIX time : 2000000000	==>	LOCAL TIME : Wed May 18 05:33:20 CEST 2033
Date string YYYY-MM-DD hh:mm:ss date into Unix timestamp :
date -d"2013-12-16 00:30:01" +%s
Unix timestamp into a formatted date string :
date -d@1387215550 "+%d/%m/%Y - %H:%M"

Compute the end of Epoch (source) :

date -d @$(echo $((2 ** 31 - 1)))

Compute the number of days since any past date :

echo "scale=3; ("$(date -d"now" +%s) - $(date -d"2016-12-18 10:05:00" +%s)") / 86400" | bc

Compute the number of days between 2 dates :

general case :
startDate='2016-12-18 10:05:00'; endDate='today'; echo "scale=1; ("$(date --date "$endDate" +%s) - $(date --date "$startDate" +%s)") / 86400" | bc
excluding start and end days :
  • startDate='yesterday'; endDate='tomorrow'; echo "scale=1; ("$(date --date "$endDate 00:00:00" +%s) - $(date --date "$startDate 23:59:59" +%s)") / 86400" | bc
  • startDate='20170330'; endDate='20170401'; printf %0.f $(echo "scale=1; ("$(date --date "$endDate" +%s) - $(date --date "$startDate" +%s)") / 86400 -1" | bc)
go even further :
startDate='2019-07-06'; endDate='2019-09-02'; nbDays=$(echo "scale=1; ("$(date --date "$endDate" +%s) - $(date --date "$startDate" +%s)") / 86400" | bc); halfNbDays=$(echo "scale=1; $nbDays/2" | bc); middleDay=$(date +"%F (%a)" --date "$startDate +$(echo $halfNbDays | cut -d'.' -f1) days"); echo -e "Nb days =|$nbDays\nHalf =|$halfNbDays\nMiddle day =|$middleDay" | column -s '|' -t
There's a hack to compute dates with integers only, which makes the whole command not exact if there's an odd number of days .

Compute an end date while still considering workdays and week-ends :

This is typically the kind of questions we face once having determined a list of tasks and their durations, and now having to plan all these.
  • I am convinced it would be possible to do this with date + Bash (believe me, I like scripting ! ), but in the light of getting-results-fast, LibreOffice Calc's workday function wins !
  • For those wondering, Excel has a similar workday function.

Travel back in time (and in logfiles) :

startYear='2014'
startMonth='08'
startDay='06'

startHour='13'
startMinute='00'
startSecond='00'

nbDaysBack=4
resolutionInHours=1	# number of hours between 2 generated datetimes

#dateTimeFormat='%d%m%y-%H'	# ddmmyy-hh
dateTimeFormat='%Y-%m-%d %H'	# yyyy-mm-dd hh

initialTimestamp=$(date -d"$startYear-$startMonth-$startDay $startHour:$startMinute:$startSecond" +%s)
nbLoops=$((24/$resolutionInHours*$nbDaysBack))
timestamp=$initialTimestamp
for i in $(seq 1 $nbLoops);do
	formattedDate=$(date -d@$timestamp +"$dateTimeFormat")
	echo $formattedDate

	# ... now do something smart with the formatted date !

	timestamp=$((timestamp-(3600*$resolutionInHours)))
done

dmidecode

DMI table decoder. To extract the whole tables, just run dmidecode as root.
The -t (or --type) flag displays the available tables : Then, to get details about a specific component : dmidecode -t processor.
It is also possible to query tables numerically (codes available with man dmidecode). Information about the motherboard (2) and on-board devices (10) : dmidecode -t 2,10

du

Usage :

Estimate disk space usage

You may face weird situations where

Read more about this here.

Flags :

Flag Usage
--apparent-size print apparent sizes, rather than disk usage; although the apparent size is usually smaller, it may be larger due to holes in ('sparse') files, internal fragmentation, indirect blocks, and the like
-c display a grand total
--max-depth depth (Linux)
-d depth (*BSD ?) 		display an entry for all files and directories depth directories deep max-depth conflicts with s
-h display results in human readable format
-m display file size in megabytes
-S --separate-dirs ignore subdirectory sizeThis REALLY ignores subdirectories !
-s summarize : display only a total for each argument s conflicts with max-depth
-x only consider files that are on the same filesystem (=device) than the file given as parameter : du -x /. This prevents going into /mnt/... filesystems.

Example :

Size of a directory :

du -sh /path/to/directory/

Size of children directories :

unsorted :
du -h --max-depth=1 /path/to/directory/
sorted from biggest to smallest :
  • du -m . --max-depth=1 | sort -nr
  • du -m . --max-depth=1 | sort -nr | head -20
sorted from biggest to smallest (alternate) :

The specified path must end with /*

du -xms ./* | sort -nr

And since sort is smart enough to sort human units, we can also run :

du -xhs ./* | sort -rh

Size of files only contained in children directories (ignoring deeper directories) :

du -S -sh * | sort -nr

Find which directory is eating megabytes on the / filesystem :

du -mx / | sort -nr | less

After copying data from a server to another, the reported "disk usages" are different. Why ? (source : 1, 2, 3)

Keep in mind that :
  • du is about the disk usage. And disk usage != sum of file sizes.
  • du reports estimates (for byte-count precision of disk usage, consider --apparent-size)
It is entirely possible that the same set of files use different amount of disk space. This is because of the intricacies of the file system (including block size, internal fragmentation, ...) (source, details).