Bash Index : T - The 'T' Bash commands : description, flags and examples

traceroute

Usage :

print the route packets trace to network host

Flags :

Flag Usage
-T Use TCP SYN for probes

tput

Usage :

initialize a terminal or query terminfo database

Example :

How can I get a terminal window's width and height ? (source)

echo "cols x lines : $(tput cols) x $(tput lines)"
cols x lines : 126 x 48

tcptraceroute

Usage :

A traceroute implementation using TCP packets :
tcptraceroute 123.45.67.89 443
Install it with tcptraceroute.
Isn't this redundant with traceroute's -T ?

timeout

Usage :

Run a command with a timeout :

timeout options duration command

This returns command's exit code if no timeout occurred. 124 otherwise (details).

Flags :

Flag Usage
duration duration can be specified as ns, nm, nh nd for seconds / minutes / hours / days. Defaults to seconds.
-k extraDuration
--kill-after=extraDuration
Also send a kill signal extraDuration time units after sending the initial signal if [command] is still running.
-s signal
--signal=signal
Send the signal signal after duration.
signal can be specified either by its name or number, defaults to TERM.

Exit Status :

Condition Exit code
timeout ? extra option
No (none) exit code of command
No --preserve-status exit code of command
Yes (none) 124
Yes --preserve-status command-dependant ?

timeoutSeconds=3; for sleepDuration in 2 5; do for option in '' '--preserve-status'; do echo -e "\nSleep : ${sleepDuration}s (timeout : ${timeoutSeconds}s), timeout option : '$option'"; /usr/bin/time -f %e timeout $option ${timeoutSeconds}s sleep $sleepDuration; echo $?; done; done

Sleep : 2s (timeout : 3s), timeout option : ''
2.00
0

Sleep : 2s (timeout : 3s), timeout option : '--preserve-status'
2.00
0

Sleep : 5s (timeout : 3s), timeout option : ''
Command exited with non-zero status 124
3.00
124

Sleep : 5s (timeout : 3s), timeout option : '--preserve-status'
Command exited with non-zero status 143
3.00
143
Explicitly using /usr/bin/time because Bash also has a time built-in command, with different options and output.

Example :

find during 3 seconds then stop, and kill after the 4th second if need be :

timeout -k 1 -s HUP 3s find /

type

Usage :

Display information about commands :

type [options] command

It can be used to :

Flags :

Flag Usage
-a display all locations containing an executable named command, includes aliases, builtins, and functions
-t display a single word describing the type of object command is : alias, keyword, function, builtin, file or nothing if not found.

Example :

type type
type is a shell builtin
type ln
ln is /bin/ln
type -a ls
ls is aliased to `ls --color=auto'
ls is /bin/ls
type -t ln
file

Check whether a user-defined function exists (source)

myFunction() { echo 'I am a function'; }; for functionName in myFunction aFunctionThatDoesNotExist; do type -t "$functionName" | grep -q 'function' && echo "'$functionName' is a function" || echo "'$functionName' does not exist"; done
'myFunction' is a function
'aFunctionThatDoesNotExist' does not exist

tune2fs

Usage :

List / adjust tunable filesystem parameters on ext2 / ext3 / ext4 filesystems.

Flags :

Flag Usage
-l device list the contents of the filesystem superblock : type, last mount point, status, FS creation/last mount/last write/last check/next check, mount count, ...

true

Do nothing, and succeed
true only returns a Unix-success exit code : 0.
: is a synonymous of true.

tree

tree is a utility designed to display directories and files tree structure. It can easily be installed on Debianoids with : apt-get install tree

Flag Usage
-d List directories only
-f Print the full path prefix for each file
-L n Go deep up to n levels maximum

Alternate shell-based solution (source) :

This solution only lists directories/subdirectories.

  • ls -R /top/directory | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'
  • ls -R /top/directory | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^-/ |/'

/top/directory won't appear on the output if it ends with a /.

How does this work ?
ls -R /top/directory
recursively list the contents of /top/directory
grep ":$"
in ls output, directories are displayed with a trailing :. This keeps directories only.
sed -e 'sedCommand' -e 'otherSedCommand' -e '...'
-e is used to chain sed commands.
sed -e 's/:$//'
remove the trailing :
-e 's/[^-][^\/]*\//--/g'
replace every path depth level with --
-e 's/^/ /'
add a leading [space]
-e 's/-/|/'
replace the leading - with a | to draw a tree

trap

Usage :

trap is designed to catch one or more signals and run the configured command(s) :

trap command(s) signal(s)

If a signal is 0 or EXIT, the commands are executed when the shell exits.

Example :

Run cleaning actions at the end of a script :

finish() {
	# clean your mess before leaving
	}

trap finish EXIT

Run specific action when CTRL-c is pressed :

trap "{ echo 'CTRL-C detected... Bye-bye.'; exit 1; }" SIGINT

tr

Usage :

tr does character substitution from standard input to standard output. In a loop over a directory containing dscnxxxx.jpg and truc.sh files, tr 'dscn' 'foto' gives :

Flags :

Flag Usage Example
-c set1 set2 Use the complement of set1 (i.e. "all characters except those in set1) (details)

I can see no use case of -c alone. tr -cd listOfCharacters can be used to keep only the listed characters.

echo 'Hello World' | tr -cd [:lower:]
elloorld
-d character
--delete character
Delete all occurrences of character from the input stream
  • DON'T : echo '2018/05/29' | tr '/' ''
    tr: when not truncating set1, string2 must be non-empty
  • DO : echo '2018/05/29' | tr -d '/'
    20180529
echo 'Hello World' | tr -d o
Hell Wrld
-s character
--squeeze-repeats character
Replace each input sequence of repeated character's with a single occurrence of character echo 'aaabbbaaaccc' | tr -s a
abbbaccc

Example :

Convert any string to lowercase (sources : 1, 2) :

echo 'This is a MiXeD-CaSe string' | tr '[:upper:]' '[:lower:]'

Remove line endings :

Replace [newline] with [space] :
tr '\n' ' ' < inputFile
Same as above + write to outputFile :
tr '\n' ' ' < inputFile > outputFile
Delete line endings (which will turn the input into a single giant line) :
  • echo -e 'foo\nbar\nbaz' | tr -d '\n'
    foobarbaz
  • tr -d '\n' < inputFile
This removes ALL line endings, including the last one. This could mislead further operations such as line count :
echo -e 'a\nb\nc' | wc -l
3
echo -e 'a\nb\nc' | tr -d '\n' | wc -l
0

Convert line endings from DOS format (CRLF aka \r\n) into Unix format (LF aka \n) (details on line endings) :

tr -d '\r' < fileToConvert > convertedFile

When tr complains tr: range-endpoints of `+-*' are in reverse collating sequence order (source) :

When finding a string like anyCharacterStart-anyCharacterStop, tr considers it as a range of characters. However, for a range to be valid, anyCharacterStart must be found prior to anyCharacterStop in the current collation order (here : ASCII for +-*).
Specifying *-+ isn't even a range since, in ASCII order, we have :
  1. *
  2. +
  3. ,
  4. -
So *-+ just means the * and + characters.

This is no big deal unless you mean * and + and -. In such case, specify *+-.

top

Usage :

List running processes in real time
top screenshot

Flags :

Shell flags

Flag Usage
-p PID Monitor only the selected PID. To specify multiple PIDs :
  • top -ppid1 -ppid2
  • top -ppid1, pid2

Interactive commands

Key Usage
< / > Move sort field next col left / right respectively (looks useless so far ... ???)
1 Toggle single/separate CPU states
b switch to reverse video to highlight running processes
B switch to bold display to highlight running processes
c toggle the long-command line display
f open the field selection window.
F or O open the Field sorting window.
i toggle idle and zombie processes
k kill a process (PID expected next)
n + number
# + number
show number processes. 0 for no limit
o open the column ordering window. This controls the order of columns from left to right on the screen, not the sorting of processes.
q quit
r renice a process (PID expected next)
R Toggle reverse sorting order
u select a user (prompts for user name)
W Write the current settings to a file
x toggle highlight of the sorting column once in b or z mode
z toggle between monochrome/color display and highlight running processes
or SPACE force task list refresh

Details on the Process Status column

  • D : uninterruptible sleep (sleep state that cannot handle a signal, such as waiting for disk or network I/O (details))
  • R : runnable (on run queue)
  • S : sleeping
  • T : traced or stopped
  • Z : zombie (defunct) (find zombies)

Details on the CPU(s) line (source) :

  • us : Percentage of CPU time spent in user space
  • sy : Percentage of CPU time spent in system (=kernel) space
  • ni : Percentage of CPU time spent in low priority (=nice) processes
  • id : Percentage of CPU time spent idle
  • wa : Percentage of CPU time spent waiting for I/Os (disk, network)
  • hi : Percentage of CPU time spent in handling hardware interrupts
  • si : Percentage of CPU time spent in handling software interrupts
  • st : steal time : the amount of CPU 'stolen' from this virtual machine by the hypervisor for other tasks (such as running another virtual machine).

Details on the memory-related columns : VIRT, RES and SHR :

Numbers are given in kb. View description of VIRT, RES and SHR columns.

Show swap usage (source) :

  1. top + f (toggle visible fields)
  2. browse down to SWAP
  3. d (toggle display)
  4. While still on the SWAP line, you can enable sorting by swap usage with s (which will update the 1st line : ...whose current sort field is SWAP)
  5. Then leave with q
  6. On the processes page, use < / > to sort asc / desc
How Linux manages the swap :
Linux swaps data from RAM based on how likely it is (in Linux's personified opinion) that it will need to be referred to any time soon. You can alter your swappiness to increase/decrease the likelihood of this happening.
To say it simple - kernel tries to use RAM and swap the way it thinks it is going to be efficient : the page/swap is not simply a buffer for when all RAM is consumed. Memory manager utilizes it in complex ways and having stuff in the swap file before RAM is filled is not a bad thing : swap usage while there is still free RAM doesn't necessarily mean your system is working slower than it could.

To forbid swapping for a specified process : echo 0 > /proc/pid/vm/swappiness

Show processes in IOWait :

List processes by descending %MEM :

top 3.2.8 :
top + F + n (the %MEM field) + Any
top 3.3.3 :
top + f + scroll to the %MEM field + s (select. Notice the 1st line of display : current sort field is %MEM) + q

The "top" & monitoring commands

top
Monitors processes, load average, uptime
htop
Advanced version of top, monitors processes, load average, uptime
iostat
Report CPU and I/O statistics
iotop
Monitors I/O usage by processes or threads
iftop
Monitors bandwidth usage on an interface : (as root) iftop -i eth0
innotop
Monitors MySQL's InnoDB activity.

touch

Usage :

Update both the access and modification times of a file to the specified time.

Flags :

Flag Usage
-a myFile Change myFile's access time to the current value, or to the time specified with -t.
-m myFile Change myFile's modification time to the current value, or to the time specified with -t.
-r referenceFile Use times of referenceFile
-t timestamp Specify time to use instead of current time. Timestamp format : [[CC]YY]MMDDhhmm[.SS].
Even though the seconds are optional, it may look dubious if many files ALL have times set to the exact minute : hh:mm:00
Seconds can be displayed with :

Example :

Specify the change time :

touch -t 201001020935.44 myFile

Update both the access and modification times of myFile :

To do so, either use both flags -a and -m, or none :
  • touch myFile
  • touch -am myFile

Touch myFile so that it appears to have been modified at the same date and time than referenceFile :

touch -r referenceFile myFile

time

Usage :

Time a command.
There are 2 time commands :
  • the Bash built-in time
  • /usr/bin/time
They have different options and output formats. If you're receiving a bash: -f: command not found error message, it must be because you're running :
  • the Bash built-in : time -f %e pwd
  • instead of : /usr/bin/time -f %e pwd

Flags :

Flag Usage
-v --verbose displays each available piece of information on the program's resource use on its own line
This only works when calling it with the absolute path : /usr/bin/time -v command. This is because Bash has a built-in time command that has less options.

Example :

Time a single command :

time myCommand

Time a compound command :

  • time { compoundCommand; } (details) Mind the spaces surrounding the curly brackets !
  • time sh -c 'compoundCommand' (details)

Retrieve duration in a shell variable :

duration=$((time { sleep 0.33; sleep 0.33; }) 2>&1 | sed -nr '/real/ s/.*m([0-9\.]+)s/\1/p'); echo "DURATION = ${duration}s"

How to grep the output of time (source) :

  • time true | grep real
    real	0m0.004s
    user	0m0.003s
    sys	0m0.001s
    grep seems to do nothing . This is because time outputs to stderr. Let's redirect the output :
  • time true 2>&1 | grep real
    real	0m0.001s
    user	0m0.002s
    sys	0m0.000s
    Still no luck. What's wrong this time (no pun intended ) is that we're actually redirecting the output of the "timed" command, instead of the output of time itself. Check it :
  • time ls 1>/dev/null
    real	0m0.003s
    user	0m0.000s
    sys	0m0.003s
    so we need to "catch" the output of time and redirect it to stdout before feeding grep :
  • { time true; } 2>&1 | grep real
    real	0m0.000s

tee

Usage :

Duplicate standard input :

someCommand | tee someFile

tee reads the standard input (here, the output of someCommand), and sends it to someFile and to the standard output. It's like a "T" on plumbering pipes (tee likes working with |'s )

Flags :

Flag Usage
-a append the output to the file (like >>) instead of overwriting / creating a new file (like >)

Example :

tail + grep + tee :

tail -f test.log | egrep --line-buffered "error" | tee -a test.log_SHORT

Hack : "echo" into several files at once :

this fails :
echo hello > myFile{1..5}
bash: myFile{1..5}: ambiguous redirect
this works :
for i in {1..5}; do echo hello > "myFile$i"; done
(no output)
this works even better :
echo hello | tee myFile{1..5}
hello
this writes into files in >-mode (details)

tcpdump

Usage :

tcpdump [options] [expression]

Flags :

Flag Usage
-D List available interfaces
-i interface Listen on interface
-L List the known data link types for the interface and exit
tcpdump -i eth0 -L
Data link types for eth0 (use option -y to set):
	DOCSIS (DOCSIS) (printing not supported)
	EN10MB (Ethernet)
-n show addresses and ports numbers numerically
-q quiet mode
-sn capture up to n bytes per packet. Defaults to 68. Setting this to 0 captures full packets
-v -vv-vvv Increasing verbosity
-w capture.pcap write the raw packets to capture.pcap rather than parsing and printing them out
-X
-XX
Show the packet's contents in hex and ASCII
Same as above, but also shows the ethernet header.

Example :

View SSH traffic to/from host.example.com :

tcpdump -XXvv -i eth0 'host host.example.com and tcp port 22'

Capture traffic coming from 172.20.75.28 :

tcpdump -s0 -w capture.pcap -ni any host 172.20.75.28

telnet

How to check whether a daemon is listening ?

telnet host [port]

If you get :
  • a welcome screen
  • a prompt
  • a blank screen (?)
  • anything different from a Connection Refused, a Connection timed out or an error message
this means someone's listening on the other end.
Leave with :
  • on Windows : CTRL-$
  • on Linux : ALT Gr-CTRL-°

then quit.

Send HTTP requests

  1. telnet www.google.com 80
  2. GET / HTTP/1.1
  3. Host: www.google.com
  • Requests are case-sensitive : trying get will return an HTTP 405 error
  • You have to press twice to send the request.
  • If the requested page is large, you can limit the length of the output to see the HTTP headers with : telnet host port | head -30

Can I probe UDP ports ?

No, you can't. Use nc -u ... instead.
Detailed answer (source)
UDP is a connectionless protocol, which means it basically just sends packets out to the specified destination. TCP is connection-oriented which means it establishes a connection to the other end using the 3-way handshake.
So it makes sense to apply the telnet paradigm to TCP : you open a connection to a specific host and port, you still remain connected (for a period of time) even if you aren't sending any data and you can send and receive data continuously without having to reconnect in-between.
UDP, on the other hand, doesn't really fit the telnet model : its more of a fire-and-forget system where you fire-off a series of packets towards the destination. You then go on with something else (or just wait doing nothing) until (or if) the remote process sends some packets back.

tail

Usage :

Display the tailing part of files.

Flags :

Flag Usage
  • -m file
  • -n m file
  • -n +m file
  • display the m tailing lines of file (short syntax)
    this doesn't work when applied to multiple files :
    tail -m file1 file2
    tail: option used in invalid context -- m
  • long syntax
  • display all tailing lines starting from the mth line of the file.
    To remove the header line :
    tail -n +2 file
-f follow : display data as it arrives into the file
-F follow with retry : try to re-open the file, even though it became inaccessible

Example :

Keep only the end of very long log files :

keepNbLines=1000; maxSizeMb=30; maxSizeBytes=$(($maxSizeMb*1024*1024)); tmpFile=$(mktemp --tmpdir tmp.XXXXXXXX); for file in $(find . -type f -size +${maxSizeBytes}c); do mv $file $tmpFile; tail -n $keepNbLines $tmpFile > $file; rm $tmpFile; done

tar

Usage :

Originally designed to make tape archives, today tar is used to gather files and directories into a single file (called a tar archive) so that moving and storing these files is more convenient. The final size of the tar archive is somewhat the sum of the size of the files it contains (unless using compression options).

There's no need to use tar for a single file : just compress it (bzip2, gzip) and send it !

Only root can create files on behalf of other users; So, in order to preserve ownership of files, you have to extract archives as root.

Flags :

Flag Usage
c create an archive which name is the next parameter
-C dir --directory=dir change the working directory to dir after that point in a list of file names (details)
--delete delete a file from a tar archive (source)
Example : tar --delete -f myArchive.tar fileToRemove_1 fileToRemove_2 ... fileToRemove_n
f next parameter is a file name (to be used almost everytime)
h --dereference don't archive symlinks themselves but the files they point to : tar -cf myArchive.tar * --dereference
By default, symlinks are archived... as symlinks
j use bzip2 as the compression algorithm
This flag is actually optional when un-archiving, read note about -Z, -z and -j flags.
o --no-same-owner When extracting an archive, do not attempt to preserve the owner specified in the tar archive.
This the default behavior for ordinary users.
p
--preserve-permissions
--same−permissions
extract information about file permissions (i.e. ignore umask when extracting files)
This is the default for root.
r append file(s) to an existing archive
t list content of archive (doesn't work on compressed archives)
v verbose
x extract files from specified archive. Files keep their original timestamp
z zip
  • compress while creating archive (when associated to c)
  • decompress while extracting archive (when associated to x)
This flag is actually optional when un-archiving, read note about -Z, -z and -j flags.
- I've noticed sometimes a tar -??? files... command doesn't work whereas tar ??? files... do. Just don't know why... (POSIX legacy stuff ?)
--exclude=pattern don't archive files matching pattern.
pattern is a shell pattern, not a RegExp.

About the -Z, -z and -j flags (source) :

As said in the docs (zgrep -E 'version 1\.15.*2004-12-20' -A 5 /usr/share/doc/tar/NEWS.gz, but nowhere in the man ) :
version 1.15 - Sergey Poznyakoff, 2004-12-20

* Compressed archives are recognised automatically, it is no longer
necessary to specify -Z, -z, or -j options to read them.  Thus, you can
now run 'tar tf archive.tar.gz'.
Let's check this :
archiveName='./archive'; for option in z j; do echo -e "\nTesting with '-$option'"; for i in hello world; do echo "$i" > "$i"; done; tar cf$option "$archiveName" hello world; rm hello world; file "$archiveName"; tar xf "$archiveName"; ls -l; rm "$archiveName" hello world; done
Testing with '-z'
./archive: gzip compressed data, last modified: Tue May 14 08:02:09 2019, from Unix		archive type reported by file
total 12
-rw------- 1 stuart developers 142 May 14 10:02 archive
-rw------- 1 stuart developers   6 May 14 10:02 hello					files properly un-archived
-rw------- 1 stuart developers   6 May 14 10:02 world

Testing with '-j'
./archive: bzip2 compressed data, block size = 900k						archive type reported by file
total 12
-rw------- 1 stuart developers 143 May 14 10:02 archive
-rw------- 1 stuart developers   6 May 14 10:02 hello					files properly un-archived
-rw------- 1 stuart developers   6 May 14 10:02 world
As a conclusion : these flags were already optional when I learned using tar, years ago .

Example :

Append file(s) to an existing archive

tar -rf archiveName.tar file1 file2 ... file_n

Build a compressed .tar package (.tar.gz, .tar.bz2)

  • 1st method :
    1. tar -cf archiveName.tar file1 file2 ... file_n to build the archive (= package of several files in a single file)
    2. gzip archiveName.tar to compress the archive
  • 2nd method : tar cfz archiveName.tgz file1 file2 ... file_n
  • 3rd method : tar -cjf archiveName.bz2 file1 file2 ... file_n

List the contents of a .tar package

tar tf archiveName.tar

Create an archive excluding the CVS files

tar cf archiveName.tar directory_1/ directory_2/ prefix_* --exclude='.svn' --dereference
  • It is also possible to specify a list of exclude patterns in an external file with --exclude-from=file (details)
  • As of tar version 1.23, the flag --exclude-vcs excludes ALL version control system files automatically (source)

Extract a single file from an archive (source)

tar -xf archiveName.tar fileToExtract

When trying to extract several archives at once using *, tar complains Not found in archive (source) :

Reproduce the problem : touch {a,b,c}.tar; tar xf *tar, outputs :

tar: This does not look like a tar archive this is normal as these are not genuine tar archives
tar: b.tar: Not found in archive
tar: c.tar: Not found in archive
tar: Exiting with failure status due to previous errors

What causes this behavior is that the command :

tar xf *tar
is interpreted as :
tar xf a.tar b.tar c.tar
by shell expansion, which is in turn interpreted by tar as "Please extract b.tar and c.tar from a.tar" (remember this ?), which can NOT work since files are not organized this way.

Solutions :
  • Use as many tar commands as the number of archives to extract : tar xf a.tar; tar xf b.tar; tar xf c.tar
  • A for loop : for archive in a.tar b.tar c.tar; do tar xf "$archive"; done
  • ls *tar | xargs tar xf (Works but this is BAD as you ought not parse the output of ls)
  • find . -type f -name "*tar" -exec tar xf {} \;
    find . -type f -name "*tar.gz" -exec tar zxf {} \;