Bash Index : T - The 'T' Bash commands : description, flags and examples

mail

timedatectl

Usage

Flags

Flag Usage
  • (none)
  • status
Show current settings of the system clock and RTC, including whether network time synchronization through systemd-timesyncd.service is active (even if it is inactive, a different service might still synchronize the clock).
list-timezones
  • (explicit)
  • there are 597 timezones so far, so you may want to grep to find what you're looking for :
    timedatectl list-timezones | grep Europe
set-timezone timezone
  • (explicit)
  • It takes a few seconds before date and the desktop clock show the corresponding date and time.
timesync-status
  • show current status of systemd-timesyncd.service
  • with --monitor : monitor the status updates
mail

traceroute

Usage

print the route packets trace to network host

Flags

Flag Usage
-T Use TCP SYN for probes
mail

tput

Usage

initialize a terminal or query terminfo database

Example

How can I get a terminal window's width and height ? (source)

echo "cols x lines : $(tput cols) x $(tput lines)"
cols x lines : 126 x 48
mail

tcptraceroute

Installed with the Debian package

tcptraceroute

Usage

A traceroute implementation using TCP packets :
tcptraceroute 12.34.56.78 443
Isn't this redundant with traceroute's -T ?
mail

timeout

Usage

Run a command with a timeout :

timeout options duration command

This returns (details) :

Flags

Flag Usage
duration duration :
  • can be specified as ns, nm, nh nd for seconds / minutes / hours / days
  • defaults to seconds
-k extraDuration
--kill-after=extraDuration
Also send a kill signal extraDuration time units after sending the initial signal if command is still running.
-s signal
--signal=signal
  • send the signal signal after duration
  • signal :
    • can be specified either by its name or number
    • defaults to TERM

Exit Status

Condition Exit code
timeout ? extra option
No (none) exit code of command
No --preserve-status exit code of command
Yes (none) 124
Yes --preserve-status command-dependant ?

timeoutSeconds=3; for sleepDuration in 2 5; do for option in '' '--preserve-status'; do echo -e "\nSleep : ${sleepDuration}s (timeout : ${timeoutSeconds}s), timeout option : '$option'"; /usr/bin/time -f %e timeout $option ${timeoutSeconds}s sleep $sleepDuration; echo $?; done; done

Sleep : 2s (timeout : 3s), timeout option : ''
2.00
0

Sleep : 2s (timeout : 3s), timeout option : '--preserve-status'
2.00
0

Sleep : 5s (timeout : 3s), timeout option : ''
Command exited with non-zero status 124
3.00
124

Sleep : 5s (timeout : 3s), timeout option : '--preserve-status'
Command exited with non-zero status 143
3.00
143
Explicitly using /usr/bin/time because Bash also has a time built-in command, with different options and output.

Example

find during 3 seconds then stop, and kill after the 4th second if need be :

timeout -k 1 -s HUP 3s find /
mail

type

Usage

Display information about commands :

type [options] command

It can be used to :

Flags

Flag Usage
-a display all locations containing an executable named command, includes aliases, builtins, and functions
-t display a single word describing the type of object command is : alias, keyword, function, builtin, file or nothing if not found.

Example

type type
type is a shell builtin
type ln
ln is /bin/ln
type -a ls
ls is aliased to `ls --color=auto'
ls is /bin/ls
type -t ln
file

Check whether a user-defined function exists (source)

myFunction() { echo 'I am a function'; }; for functionName in myFunction aFunctionThatDoesNotExist; do type -t "$functionName" | grep -q 'function' && echo "'$functionName' is a function" || echo "'$functionName' does not exist"; done
'myFunction' is a function
'aFunctionThatDoesNotExist' does not exist
mail

tune2fs

Usage

List / adjust tunable filesystem parameters on ext2 / ext3 / ext4 filesystems.

Flags

Flag Usage
-l device list the contents of the filesystem superblock : type, last mount point, status, FS creation/last mount/last write/last check/next check, mount count, ...
mail

true

Do nothing, and succeed
true only returns a Unix-success exit code : 0.
: is a synonymous of true (details).
mail

tree

tree is a utility designed to display directories and files tree structure. It can easily be installed on Debianoids with : apt-get install tree

Flag Usage
-d List directories only
-f Print the full path prefix for each file
-L n Go deep up to n levels maximum

Alternate shell-based solutions :

sed-based :

  • ls -R '/path/to/directory' | grep ':$' | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'
      • only lists directories / subdirectories
      • /path/to/directory won't appear on the output if it ends with a /
    • inspired by
    • how it works :
      ls -R /path/to/directory
      recursively list the contents of /path/to/directory
      grep ':$'
      in ls output, directories are displayed with a trailing :. This keeps directories only.
      sed -e 'sedCommand' -e 'otherSedCommand' -e ''
      -e is used to chain sed commands.
      sed -e 's/:$//'
      remove the trailing :
      -e 's/[^-][^\/]*\//--/g'
      replace every path depth level with --
      -e 's/^/ /'
      add a leading [space]
      -e 's/-/|/'
      replace the leading - with a | to draw a tree (no s///g so it matches the first it finds)

awk-based :

dirToDisplay='/path/to/directory'
cd "$dirToDisplay"; ls -R | awk -F '/' '/:$/ {	\
	if (length($NF)==2)	{ next }		\
	if (NF>0)		{ printf " " }	\
	for (i=0;i<NF-2;i++)	{ printf "|   " }	\
	if (NF>1)		{ printf "+--" }	\
	cleanDirName=gensub(/:$/, "", "g", $NF); print cleanDirName \
	}'; cd - 1>/dev/null
About lines above :
if (length($NF)==2) { next }
skip lines where $NF is a single-letter directory name followed by the : appended by ls -R
mail

trap

Usage

trap is designed to catch one or more signals and run the configured command(s) :

trap commands listOfSignalsToCatch

Special behaviour on some signals

  • 0 or EXIT : run commands when the shell exits
  • DEBUG : run commands before every simple command
  • RETURN : run commands each time a function or script finishes executing
  • ERR : run commands every time a command exits with a non-zero status (except for if, while and until loops and AND &&, OR || and NOT ! logical operations)
    these situations and exceptions are those of the shell -e flag

Flags

Flag Usage
-l print a list of signal names and numbers
-p signal list traps that are currently registered with signal (example)
--help (explicit)

Example

Run cleaning actions at the end of a script :

finish() {
	# clean your mess before leaving
	}

trap finish EXIT

Run specific action when CTRL-c is pressed :

trap "{ echo 'CTRL-C detected... Bye-bye.'; exit 1; }" SIGINT

List traps :

trap "echo 'CTRL-C detected'" SIGINT; trap -p SIGINT

Clear a trap (source) :

trap - listOfSignalsToStopCatching
mail

tr

Usage

tr does character substitution from standard input to standard output. In a loop over a directory containing dscnxxxx.jpg and truc.sh files, tr 'dscn' 'foto' gives :

Flags

Flag Usage Example
-c set1 set2 Use the complement of set1 (i.e. "all characters except those in set1) (details)

I can see no use case of -c alone. tr -cd listOfCharacters can be used to keep only the listed characters.

echo 'Hello World' | tr -cd [:lower:]
elloorld
-d character
--delete character
Delete all occurrences of character from the input stream
  • DON'T : echo '2018/05/29' | tr '/' ''
    tr: when not truncating set1, string2 must be non-empty
  • DO : echo '2018/05/29' | tr -d '/'
    20180529
echo 'Hello World' | tr -d o
Hell Wrld
-s characters
--squeeze-repeats characters
Replace repetitions of any character of characters with a single occurrence of itself
echo 'aa bb cc' | tr -s a
a bb cc
echo 'aa aa bb cc' | tr -s a
a a bb cc
echo 'aa bb cc' | tr -s ab
a b cc
If 2 sets of characters are provided : tr -s set1 set2, tr :
  1. performs replacements using set1 and set2 (as if -s were not specified)
  2. then removes repetitions of characters of set2
echo 'aa bb cc' | tr -s a b
b b cc
echo 'aa bb cc' | tr a b | tr -s b
b b cc

Example

Convert any string to lowercase (sources : 1, 2) :

echo 'This is a MiXeD-CaSe string' | tr '[:upper:]' '[:lower:]'

Make SPACE characters visible :

tr ' ' '_' < someFile

Remove line endings :

Replace [newline] with [space] :
tr '\n' ' ' < inputFile
Same as above + write to outputFile :
tr '\n' ' ' < inputFile > outputFile
Delete line endings (which will turn the input into a single giant line) :
  • echo -e 'foo\nbar\nbaz' | tr -d '\n'
    foobarbaz
  • tr -d '\n' < inputFile
This removes ALL line endings, including the last one. This could mislead further operations such as line count :
echo -e 'a\nb\nc' | wc -l
3
echo -e 'a\nb\nc' | tr -d '\n' | wc -l
0

Convert line endings from DOS format (CRLF aka \r\n) into Unix format (LF aka \n) (details on line endings) :

tr -d '\r' < fileToConvert > convertedFile

When tr complains tr: range-endpoints of `+-*' are in reverse collating sequence order (source) :

When finding a string like anyCharacterStart-anyCharacterStop, tr considers it as a range of characters. However, for a range to be valid, anyCharacterStart must be found prior to anyCharacterStop in the current collation order (here : ASCII for +-*).
Specifying *-+ isn't even a range since, in ASCII order, we have :
  1. *
  2. +
  3. ,
  4. -
So *-+ just means the * and + characters.

This is no big deal unless you mean * and + and -. In such case, specify *+-.

mail

top

Usage

List running processes in real time
top screenshot

Flags

Shell flags

Flag Usage
-p PID Monitor only the selected PID. To specify multiple PIDs :
  • top -ppid1,pid2,pid3
  • top -ppid1 -ppid2 -ppid3

Interactive commands

details : man -P 'less -p "^[0-9]\. INTERACTIVE Commands"' top
Key Usage
< / > set the column on the left/right as the sorting column (unless the current column is already the left/rightmost column)
use this with x
1 toggle single/separate CPU states
b switch to reverse video to highlight running processes
B switch to bold display to highlight running processes
c toggle the long-command line display
f open the field selection window.
F or O open the Field sorting window.
i toggle idle and zombie processes
k kill a process (PID expected next)
m cycle through displays of memory+SWAP usage :
  1. KiB total / free / used / buffer-cache
  2. % + bar graph
  3. % + block graph
  4. off
n + number
# + number
show number processes. 0 for no limit
o open the column ordering window. This controls the order of columns from left to right on the screen, not the sorting of processes.
q quit
r renice a process (PID expected next)
R Toggle reverse sorting order
u select a user (prompts for user name)
W Write the current settings to a file
x toggle highlight of the sorting column
z toggle between monochrome/color display and highlight running processes
or SPACE force task list refresh

Details on the Process Status column (source : man -P 'less -p "Process Status$"' top, related : man -P 'less -p "^PROCESS STATE CODES"' ps)

D : uninterruptible sleep
  • sleep state that cannot handle a signal, such as waiting for disk or network I/O
  • details
I : idle
R : runnable
process in the run queue : ready to run or currently running
S : sleeping
T : stopped by job control signal
(traced)
t : stopped by debugger during trace
Z : zombie
  • the process itself has actually ended, but the parent process has not reclaimed its resources (PID, ...)
  • find zombies

Details on the CPU(s) line (source) :

  • us : Percentage of CPU time spent in user space
  • sy : Percentage of CPU time spent in system (=kernel) space
  • ni : Percentage of CPU time spent in low priority (=nice) processes
  • id : Percentage of CPU time spent idle
  • wa : Percentage of CPU time spent waiting for I/Os (disk, network), aka IOWait
  • hi : Percentage of CPU time spent in handling hardware interrupts
  • si : Percentage of CPU time spent in handling software interrupts
  • st : steal time : the amount of CPU 'stolen' from this virtual machine by the hypervisor for other tasks (such as running another virtual machine).

More about the IOWait value (source) :

  • IOWait is the relative percentage of time that your system could have been doing useful work if the disks/network were faster.
  • a climbing %iowait is a danger sign that your system may be running into an IO bottleneck
  • BUT a low %iowait is not necessarily an indication that you don't have an IO problem (the problem might just not occur RIGHT NOW); you also want to look at things like the number of processes shown as blocked ('b' state) in vmstat output
Example before / after a CPU upgrade :
  • Before CPU Upgrade
    • CPU time = 40 ms
    • IO time = 20 ms
    • Total transaction time = CPU + IO = 40 + 20 = 60 ms
    • %iowait = IO time/total time = 20/60 = 33%
  • After CPU Upgrade
    • CPU time = 40 ms/ 4 = 10 ms
    • IO time = 20 ms
    • Total transaction time = CPU + IO = 10 + 20 = 30 ms
    • %iowait = 20/30 = 66%
In this example :
  • transaction performance doubled, despite a 2x increase in %iowait
  • the absolute value of %iowait is a misleading indicator of I/O problems

Details on the memory-related columns : VIRT, RES and SHR :

Numbers are given in kb. View description of VIRT, RES and SHR columns.

Show swap usage (source) :

  1. top + f (toggle visible fields)
  2. browse down to SWAP
  3. d (toggle display)
  4. While still on the SWAP line, you can enable sorting by swap usage with s (which will update the 1st line : ...whose current sort field is SWAP)
  5. Then leave with q
  6. On the processes page, use < / > to sort asc / desc

How Linux manages the swap :

Linux swaps data from RAM based on how likely it is (in Linux's personified opinion) that it will need to be referred to any time soon. You can alter your swappiness to increase/decrease the likelihood of this happening.
To say it simple - kernel tries to use RAM and swap the way it thinks it is going to be efficient : the page/swap is not simply a buffer for when all RAM is consumed. Memory manager utilizes it in complex ways and having stuff in the swap file before RAM is filled is not a bad thing : swap usage while there is still free RAM doesn't necessarily mean your system is working slower than it could.

To forbid swapping for a specified process : echo 0 > /proc/pid/vm/swappiness

Show processes in IOWait :

List processes by descending %MEM :

top 3.2.8 :

top + F + n (the %MEM field) + Any

top 3.3.3 :

top + f + scroll to the %MEM field + s (select. Notice the 1st line of display : current sort field is %MEM) + q
mail

The "top" & monitoring commands

top
Monitors processes, load average, uptime
htop
Advanced version of top, monitors processes, load average, uptime
iostat
Report CPU and I/O statistics
iotop
Monitors I/O usage by processes or threads
iftop
Monitors bandwidth usage on an interface : (as root) iftop -i eth0
innotop
Monitors MySQL's InnoDB activity.
mail

touch

Usage

change file timestamps to the specified time. In more detail, touch myFile :

Flags

Flag Usage
-a myFile Change myFile's access time to the current value, or to the time specified with -t.
-d dateString --date=dateString use dateString instead of the current time. dateString is an almost "free form" human readable date string :
myFile=$(mktemp); date; for dateString in 'Sun, 29 Feb 2004 16:21:42 -0800' '2004-02-27 14:19:13.489392193 +0530' 'next Thursday' '' '@0' '@87000'; do echo -e "\n'$dateString'"; touch -d "$dateString" "$myFile"; ls -l "$myFile"; done; [ -f "$myFile" ] && rm "$myFile"
Fri 16 Oct 2020 12:03:28 PM CEST					now

'Sun, 29 Feb 2004 16:21:42 -0800'
-rw------- 1 bob users 0 Mar  1  2004 /tmp/tmp.v7ewATXc4s		due to time zones

'2004-02-27 14:19:13.489392193 +0530'
-rw------- 1 bob users 0 Feb 27  2004 /tmp/tmp.v7ewATXc4s

'next Thursday'
-rw------- 1 bob users 0 Oct 22  2020 /tmp/tmp.v7ewATXc4s

''
-rw------- 1 bob users 0 Oct 16 00:00 /tmp/tmp.v7ewATXc4s		an empty string '' means "beginning of today"

'@0'
-rw------- 1 bob users 0 Jan  1  1970 /tmp/tmp.v7ewATXc4s		Unix Epoch

'@87000'
-rw------- 1 bob users 0 Jan  2  1970 /tmp/tmp.v7ewATXc4s		@n is for "n seconds after Epoch"
-m myFile Change myFile's modification time to the current value, or to the time specified with -t.
-r referenceFile Use times of referenceFile
-t timestamp Specify time to use instead of current time. Timestamp format : [[CC]YY]MMDDhhmm[.SS].
Even though the seconds are optional, it may look dubious if many files ALL have times set to the exact minute : hh:mm:00
Seconds can be displayed with :

Example

Specify the change time :

touch -t 201001020935.44 myFile

Update both the access and modification times of myFile :

To do so, either use both flags -a and -m, or none :
  • touch myFile
  • touch -am myFile

touch myFile so that it appears to have been modified at the same date and time than referenceFile :

touch -r referenceFile myFile

touch myFile to make it younger / older by n days :

fileToTouch='myFile'; dateChangeRelativeToCurrentFileDate='-42 days'; touch "$fileToTouch"; ls -l "$fileToTouch"; fileDate_current=$(date -r "$fileToTouch"); fileDate_new=$(date -d "$fileDate_current $dateChangeRelativeToCurrentFileDate"); touch -d "$fileDate_new" "$fileToTouch"; ls -l "$fileToTouch"
-rw------- 1 stuart developers 0 Jul  7 14:13 myFile
-rw------- 1 stuart developers 0 May 26 14:13 myFile
mail

time

Usage

Time a command.
There are 2 time commands () :
  • the Bash built-in time
  • /usr/bin/time
They have different options and output formats : only /usr/bin/time has a -f option. So if you get a bash: -f: command not found error message, it must be because you're running :
  • the Bash built-in : time -f %e pwd
  • instead of : /usr/bin/time -f %e pwd
In other words : time executes the Bash version.

Flags

Flag Bash time /usr/bin/time Usage
-f formatSpec
--format formatSpec
-v --verbose displays each available piece of information on the program's resource use on its own line
This only works when calling it with the absolute path : /usr/bin/time -v command. This is because Bash has a built-in time command that has less options.

Example

Time a single command :

time myCommand

Time a compound command :

  • time { compoundCommand; } (details) Mind the spaces surrounding the curly brackets !
  • time sh -c 'compoundCommand'

Retrieve duration in a shell variable :

duration=$((time { sleep 0.33; sleep 0.33; }) 2>&1 | sed -nr '/real/ s/.*m([0-9\.]+)s/\1/p'); echo "DURATION = ${duration}s"

How to grep the output of time (source) :

  • time true | grep real
    real	0m0.004s
    user	0m0.003s
    sys	0m0.001s
    grep seems to do nothing . This is because time outputs to stderr. Let's redirect the output :
  • time true 2>&1 | grep real
    real	0m0.001s
    user	0m0.002s
    sys	0m0.000s
    Still no luck. What's wrong this time (no pun intended ) is that we're actually redirecting the output of the "timed" command, instead of the output of time itself. Check it :
  • time ls 1>/dev/null
    real	0m0.003s
    user	0m0.000s
    sys	0m0.003s
    so we need to "catch" the output of time and redirect it to stdout before feeding grep :
  • { time true; } 2>&1 | grep real
    real	0m0.000s
mail

tee

Usage

Duplicate standard input :
someCommand | tee someFile
  1. tee reads the standard input (here, the output of someCommand)
  2. and sends it to
    • someFile
    • and to the standard output
    It's like a T on plumbing pipes (which is why tee enjoys |'s )
Even though tee is typically used to write to a single file, it will actually write to as many files as instructed :
echo 'hello world' | tee myFile{1..5}; ls -1 myFile*; rm myFile{1..5}
hello world
myFile1
myFile2
myFile3
myFile4
myFile5
Many process substitutions hacks rely on this .

Flags

Flag Usage
-a append the output to the file (like >>) instead of overwriting / creating a new file (like >)

Example

tail + grep + tee :

tail -f test.log | egrep --line-buffered "error" | tee -a test.log_SHORT

Hack : "echo" into several files at once :

this fails :
echo hello > myFile{1..5}
bash: myFile{1..5}: ambiguous redirect
this works :
for i in {1..5}; do echo hello > "myFile$i"; done
(no output)
this works even better :
echo hello | tee myFile{1..5}
hello
this writes into files in >-mode (details)
mail

tcpdump

Usage

tcpdump [options] [expression]

Flags

Flag Usage
-D List available interfaces
-i interface Listen on interface
-L List the known data link types for the interface and exit
tcpdump -i eth0 -L
Data link types for eth0 (use option -y to set):
	DOCSIS (DOCSIS) (printing not supported)
	EN10MB (Ethernet)
-n show addresses and ports numerically
-q quiet mode
-sn capture up to n bytes per packet. Defaults to 68. Setting this to 0 captures full packets
-v -vv-vvv Increasing verbosity
-w capture.pcap write the raw packets to capture.pcap rather than parsing and printing them out
-X
-XX
Show the packet's contents in hex and ASCII
Same as above, but also shows the ethernet header.

Example

View SSH traffic to/from host.example.com :

tcpdump -XXvv -i eth0 'host host.example.com and tcp port 22'

Capture traffic coming from 172.20.75.28 :

tcpdump -s0 -w capture.pcap -ni any host 172.20.75.28
mail

telnet

How to check whether a daemon is listening ?

If you get :
  • a welcome screen
  • a prompt
  • a blank screen (?)
  • anything different from a Connection Refused, a Connection timed out or an error message
this means someone's listening on the other end.
Leave with :
  • on Windows : CTRL-$
  • on Linux : ALT Gr-CTRL-°

then quit.

Send HTTP requests

  1. telnet www.google.com 80
  2. GET / HTTP/1.1
  3. Host: www.google.com
  • Requests are case-sensitive : trying get will return an HTTP 405 error
  • You have to press twice to send the request.
  • If the requested page is large, you can limit the length of the output to see the HTTP headers with : telnet host tcpPort | head -30

Can I probe UDP ports ? (source)

No, you can't. Use nc -u instead.

UDP is a connectionless protocol, which means it basically just sends packets out to the specified destination. TCP is connection-oriented which means it establishes a connection to the other end using the 3-way handshake.
So it makes sense to apply the telnet paradigm to TCP : you open a connection to a specific host and port, you still remain connected (for a period of time) even if you aren't sending any data and you can send and receive data continuously without having to reconnect in-between.
UDP, on the other hand, doesn't really fit the telnet model : its more of a fire-and-forget system where you fire-off a series of packets towards the destination. You then go on with something else (or just wait doing nothing) until (or if) the remote process sends some packets back.

mail

tail

Usage

Display the tailing part of files.

Flags

Flag Usage
-n number
--lines=number
  • display the last number lines
  • if number is +number : start displaying from line number
    To remove the header line :
    tail -n +2 file
  • The short syntax :
    tail -number myFile or tail +number myFile
    • is obsolete and supported only for compatibility
    • applies to 1 file at a time (the long syntax supports many files)
    • "works" only if it doesn't conflict with the "normal" usage
    source : info '(coreutils) tail invocation'
-f follow : display data as it arrives into the file
-F follow with retry : try to re-open the file, even though it became inaccessible

Example

Keep only the end of very long log files :

keepNbLines=1000; maxSizeMb=30; maxSizeBytes=$(($maxSizeMb*1024*1024)); tmpFile=$(mktemp --tmpdir tmp.XXXXXXXX); for file in $(find . -type f -size +${maxSizeBytes}c); do mv $file $tmpFile; tail -n $keepNbLines $tmpFile > $file; rm $tmpFile; done
mail

tar

Usage

Originally designed to make tape archives, today tar is used to gather files and directories into a single file (called a tar archive) so that moving and storing these files is more convenient. The final size of the tar archive is somewhat the sum of the size of the files it contains (unless using compression options).

There's no need to use tar for a single file : just compress it (bzip2, gzip).

Only root can create files on behalf of other users. So, in order to preserve ownership of files, you have to extract archives as root.

Flags

Flag Usage
c create an archive which name is the next parameter
-C dir --directory=dir change the working directory to dir after that point in a list of file names (details)
--delete delete file(s) from a tar archive
Does not work on compressed archives.
tar f archiveName.tar --delete fileToRemove_1 fileToRemove_2 fileToRemove_n
f next parameter is a file name (to be used almost everytime)
h --dereference don't archive symlinks themselves but the files they point to : tar -cf archiveName.tar * --dereference
By default, symlinks are archived... as symlinks
j use bzip2 as the compression algorithm
This flag is actually optional when un-archiving, read note about -Z, -z and -j flags.
-L length
--tape-length=length
  • generate archive files of lengthKiB
  • it's possible to use a size suffix :
    • c : bytes
    • K, k, b : KiB
    • M : MiB
    • G, T, P : GiB, TiB, PiB
  • implies -M
-M --multi-volume create / list / extract multi-volume archive, i.e. split the generated archive. See -L
o --no-same-owner When extracting an archive, do not attempt to preserve the owner specified in the tar archive.
This the default behavior for ordinary users.
p
--preserve-permissions
--same−permissions
extract information about file permissions (i.e. ignore umask when extracting files)
This is the default for root.
r append file(s) to an existing archive
t list content of archive (doesn't work on compressed archives)
v verbose
x extract files from specified archive. Files keep their original timestamp
z use gzip :
  • to compress data when creating an archive (i.e. with c)
  • to decompress data when extracting an archive (i.e. with x)
-
I've noticed sometimes a tar -[flags] command doesn't work whereas tar [flags] does. Just don't know why... (POSIX legacy stuff ?)
--exclude='pattern' don't archive files matching pattern (details, common problems with this option)
pattern :
  • is a shell pattern, not a RegExp
  • must be quoted —so that tar actually receives the pattern— otherwise, the special characters of the pattern will be substituted by shell expansion
Looks like --exclude is expected right before the list of files to archive :
  • tar --exclude='lost?found' cjf archiveName.tbz directory/to/archive/*
    tar: You must specify one of the '-Acdtrux', '--delete' or '--test-label' options
  • tar cjf --exclude='lost?found' archiveName.tbz directory/to/archive/*
    tar: archiveName.tbz: Cannot stat: No such file or directory
    tar: directory/to/archive/*: Cannot stat: No such file or directory
    tar: Exiting with failure status due to previous errors
  • tar cjf archiveName.tbz directory/to/archive/* --exclude='lost?found'
    tar: directory/to/archive/lost+found: Cannot open: Permission denied
  • tar cjf archiveName.tbz --exclude='lost?found' directory/to/archive/*
    (works fine)

About the -Z, -z and -j flags (source) :

As said in the docs (zgrep -E 'version 1\.15.*2004-12-20' -A 5 /usr/share/doc/tar/NEWS.gz, but nowhere in the man ) :
version 1.15 - Sergey Poznyakoff, 2004-12-20

* Compressed archives are recognised automatically, it is no longer
necessary to specify -Z, -z, or -j options to read them. Thus, you can
now run 'tar tf archive.tar.gz'.

Let's check this :

archiveName='./archive'; for option in z j; do echo -e "\nTesting with '-$option'"; for i in hello world; do echo "$i" > "$i"; done; tar cf$option "$archiveName" hello world; rm hello world; file "$archiveName"; tar xf "$archiveName"; ls -l; rm "$archiveName" hello world; done
Testing with '-z'
./archive: gzip compressed data, last modified: Tue May 14 08:02:09 2019, from Unix		archive type reported by file
total 12
-rw------- 1 stuart developers 142 May 14 10:02 archive
-rw------- 1 stuart developers   6 May 14 10:02 hello					files properly un-archived
-rw------- 1 stuart developers   6 May 14 10:02 world

Testing with '-j'
./archive: bzip2 compressed data, block size = 900k						archive type reported by file
total 12
-rw------- 1 stuart developers 143 May 14 10:02 archive
-rw------- 1 stuart developers   6 May 14 10:02 hello					files properly un-archived
-rw------- 1 stuart developers   6 May 14 10:02 world
As a conclusion : these flags were already optional when I learned using tar, years ago .

Example

Append file(s) to an existing archive

tar -rf archiveName.tar file1 file2 ... file_n

Build a compressed .tar package (.tar.gz, .tar.bz2)

  • 1st method :
    1. tar -cf archiveName.tar file1 file2 ... file_n to build the archive (= package of several files in a single file)
    2. gzip archiveName.tar to compress the archive
  • 2nd method : tar cfz archiveName.tgz file1 file2 ... file_n
  • 3rd method : tar -cjf archiveName.bz2 file1 file2 ... file_n

List the contents of a .tar package

tar tf archiveName.tar

Create an archive excluding the CVS files

tar cf archiveName.tar directory_1/ directory_2/ prefix_* --exclude='.svn' --dereference
  • It is also possible to specify a list of exclude patterns in an external file with --exclude-from=file (details)
  • As of tar version 1.23, the flag --exclude-vcs excludes ALL version control system files automatically (source)

Extract a single file from an archive (source)

tar -xf archiveName.tar fileToExtract

When trying to extract several archives at once using *, tar complains Not found in archive (source) :

Reproduce the problem : touch {a,b,c}.tar; tar xf *tar, outputs :

tar: This does not look like a tar archive this is normal as these are not genuine tar archives
tar: b.tar: Not found in archive
tar: c.tar: Not found in archive
tar: Exiting with failure status due to previous errors

What causes this behavior is that the command :

tar xf *tar
is interpreted as :
tar xf a.tar b.tar c.tar
by shell expansion, which is in turn interpreted by tar as "Please extract b.tar and c.tar from a.tar" (remember this ?), which can NOT work since files are not organized this way.

Solutions :

  • Use as many tar commands as the number of archives to extract : tar xf a.tar; tar xf b.tar; tar xf c.tar
  • A for loop : for archive in a.tar b.tar c.tar; do tar xf "$archive"; done
  • ls *tar | xargs tar xf (Works but this is BAD as you ought not parse the output of ls)
  • find -type f -name "*tar" -exec tar xf {} \;
    find -type f -name "*tar.gz" -exec tar zxf {} \;