Bash Index : L - The 'L' Bash commands : description, flags and examples

lame

Flags :

Flag Usage
-b set bitrate
-v enable variable bitrate

Example :

Encode *wav into *mp3 :

for i in *wav; do lame -v -b 256 "$i" "$i.mp3"; done

logger

Usage :

enter messages into the system log

I considered this after reading answers / comments to this question :

You are learning why using standard output and redirecting it to a log file is just about the worst possible approach to logging. Your log file is tied to the logging process, and you can't do anything about it. The actual solution is to use a proper logging system that would allow you to separate the actual log file from the process and allow you to use tools like logrotate. There are a lot of really good reasons things like syslog and logrotate exist —and you're learning a lot of them.

Example :

My first steps with logger :

Create a new log entry :
logger 'hello world'
Find it :
journalctl -r _COMM=logger
-- Logs begin at Thu 2017-11-02 08:27:51 CET, end at Tue 2019-04-02 16:15:41 CEST. --
Apr 02 16:15:41 myLaptop bob[28892]: hello world
...

ldd

Usage :

print shared library dependencies
  • In the usual case, ldd invokes the standard dynamic linker with the LD_TRACE_LOADED_OBJECTS environment variable set to 1, which causes the linker to display the library dependencies.
  • Be aware, however, that in some circumstances, some versions of ldd may attempt to obtain the dependency information by directly executing the program. Thus, you should never employ ldd on an untrusted executable, since this may result in the execution of arbitrary code. A safer alternative when dealing with untrusted executables is :
    objdump -p /path/to/program | grep NEEDED

let

Usage :

let is a shell built-in function to evaluate one or more arithmetic expressions.
The shell can perform arithmetic evaluations under the influence of Details : man bash + search ARITHMETIC EVALUATION.

lsblk

Usage :

lsblk lists information about all available or the specified block devices. It reads the sysfs filesystem and udev database to gather information.

lsblk -fap

Flags :

Flag Usage
-a --all List all devices, including empty one (they are skipped by default)
-f --fs Output info filesystems : name (label), FS type, UUID, mount point
-l --list Format output as a list rather than a tree
-p --paths Print full device path, e.g. /dev/mapper/vg-root rather than just vg-root
-S --scsi Output info about SCSI devices only

local

By default, variables defined in scripts are global.
local myVariable
makes myVariable's value visible only within the block of code in which it appears.
#!/usr/bin/env bash

number=0

myFunction1() {
	number=1
	echo "myFunction1 : $number"
	}

myFunction2() {
	local number=2
	echo "myFunction2 : $number"
	}

echo $number
myFunction1
echo $number
myFunction2
echo $number
0			global initial value
myFunction1 : 1		myFunction1 internal value
1			global value overwritten by myFunction1
myFunction2 : 2		myFunction2 internal value
1			unchanged global value

ln

Usage :

Make links between files : ln target linkName

Differences between hard and soft (or "symbolic") links (details) :

hard links :
  • share a single inode with their target.
  • can not refer to a missing file or to a directory. The target must be an existing regular file.
  • can only refer to a file found on the same filesystem (because inodes are not unique across filesystems).
  • How to spot a hard link ?
symbolic links :
  • can refer to directories, missing files, another symbolic link, and even cross file system boundaries.
  • have their own inode.

Interesting facts about symlinks (source) :

  • On Linux, the permissions of a symbolic link link are not used in any operations. The permissions are always 0777 and can't be changed.
  • You can change the owner of a symbolic link with chown -h.
  • when most operations (opening, reading, writing, and so on) are passed the symbolic link file, the kernel automatically "dereferences" the link and operates on the target of the link. But some operations (e.g., removing) work on the link file itself, rather than on its target.
  • Most commands dereference (i.e. follow) symbolic links, with the notable exception of :
    • mv
    • rm
    • file
    • ls
    • and MANY others when using their recursive (-R) flag

About symlinks exploits (details) :

There was a vulnerability exploiting symbolic links to overwrite arbitrary files (details). This has been fixed, and symbolic links can be followed only when :
  • outside a sticky world-writable directory. Sticky world-writable directories, like /tmp, have drwxrwxrwt permissions
  • OR when the UID of the symlink and follower match
  • OR when the directory owner matches the symlink's owner

This protection is enabled by :

echo 1 > /proc/sys/fs/protected_symlinks
(default setting on Debian Wheezy). It protects root too.
it is still recommend to create temporary files with mktemp.

Flags :

Flag Usage
(none) Create a hard link
-f --force Overwrite link if it already exists
When man ln says :
-f, --force
	remove existing destination files
what is meant by a destination file is the file representing the link itself (link in the example below), not the target of the link (i.e. not target).
Check it :
cd /tmp; touch target; echo '1. create link'; ln -s target link; echo '2. try to overwrite link'; ln -s target link; echo '3. overwrite link'; ln -sf target link; rm link target
1. create link
2. try to overwrite link
ln: creating symbolic link `link': File exists
3. overwrite link
-s Create a symbolic link (a.k.a "symlink")

last

Usage :

Shows listing of last logged in users. Extracts information from /var/log/wtmp.

The pseudo user reboot logs in each time the system is rebooted. Thus last reboot will show a log of all reboots since the log file was created.

last / lastb :

lastb is the same as last (lastb symlinks to /usr/bin/last), except that by default it shows a log of /var/log/btmp, which contains all the bad login attempts.

On Debian Wheezy, /var/log/btmp does not trace failed SSH login attempts. This is a known Debian bug, and has been fixed with OpenSSH package version 1:6.6p1-1 (source).

Flags :

Flag Usage
-number -n number Show latest number entries
-F Full display : login time + logout time + remote host
-i Display IP addresses rather than hostnames
-x Display the system shutdown entries and runlevel changes

locale

Usage :

Get locale-specific information.

Flags :

Flag Usage
(none) Display a summary of the current locale environment
-a List all available locales

Example :

In case keyboard is f*cked up after playing with locales (and doesn't accept éèà... anymore) (see also : How to reconfigure locales) :

export LANG="fr_FR.UTF-8"

During a PuTTY session, no effect as normal user, worked as root not in the current shell but in a new one.

When the system complains at any time : perl: warning: Setting locale failed." (sources : 1, 2) :

The full error message was :
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LANG = "fr_FR.UTF-8"
	are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

There may be unnecessary / overkill / wrong(?) steps in the solution below.

  1. Contrary to popular belief, locale-gen takes NO argument (man locale-gen, if you don't believe me). So our first step is to select the locales we want by uncommenting them from /etc/locale.gen
  2. Then rebuild locales : locale-gen
  3. dpkg-reconfigure locales, leave everything as-is on the first page, then set the system language to en_US.UTF-8
  4. export LANG="en_US.UTF-8"

ls

Usage :

list files (in Unix, everything is a file )

It is considered a BAD practice to parse the output of ls because file names may include special characters (space, newline, ...) that can interfere with detecting where a file name starts and ends (details). Commands such as :

should be banned.

To list a HUGE number of files, consider the -f and -U options below, or this dedicated C program.

There may be situations where ls reports a file being a given size (possibly huge) whereas it's actually way smaller. This can happen with files actually containing file systems and being dynamically provisioned (and, generally speaking, any flavor of sparse files). In such case, to get the _actual_ file size :

A very interesting answer to the "cp file1 file2 vs cat file1 > file2" question

Flags :

Flag Usage
-1 list 1 file per line
-a --all list all files, including hidden files (name starting with a .)
-A --almost-all do not list implied . and ..
-b --escape print octal escapes for nongraphic characters
--block-size=size With -l : scale sizes by size before printing them. size is an integer and optional unit (example: 10K is 10*1024). Units are :
  • powers of 1024 : K, M, G, T, P, E, Z, Y
  • powers of 1000 : KB, MB, ...
Files smaller than size will actually be reported 1 unit :
tmpFile=$(mktemp); fallocate -l 12m "$tmpFile"; for unit in K M G T; do echo "unit : $unit"; ls -l --block-size=1$unit "$tmpFile"; done; rm "$tmpFile"
unit : K
-rw------- 1 kevin users 12288 Mar 7 09:41 /tmp/tmp.NdDV69WZth	KiB : OK
unit : M
-rw------- 1 kevin users 12 Mar 7 09:41 /tmp/tmp.NdDV69WZth	MiB : OK
unit : G
-rw------- 1 kevin users 1 Mar 7 09:41 /tmp/tmp.NdDV69WZth		GiB : KO
unit : T
-rw------- 1 kevin users 1 Mar 7 09:41 /tmp/tmp.NdDV69WZth		TiB : KO
-c
  • with -lt : sort by ctime, show ctime
  • with -l : sort by name, show ctime
  • otherwise : sort by ctime, newest first
--color=when colorize the output. when is one of :
  • always (default)
  • auto
  • never
ls colors are often automatically disabled in compound commands (pipes or find ... -exec ls ...). They can be enabled by being explicitly specified in the command line with --color=always
-d --directory list directories themselves, not their contents
-F --classify append indicator (one of *, /, =, >, @, |) to entries (for details about these : info ls + search classify)
I see no real use case for this option (especially if you have enabled colored output), which may display misleading information : is someFile* a filename actually ending with a * or a file with the execution bit set ?
I don't recommend using this option.
-f do not sort + enable -aU + disable -ls and --color (=--color=never)
-h --human-readable print file size in human readable format (2K, 34M, 1G, ...)
-H
--dereference-command-line
follow symbolic links listed on the command line
symlinks MUST be specified on the command line to be de-referenced :
ls -H /etc/apache2/sites-enabled/
Will fail. Actually, -H will have no effect here
ls -H /etc/apache2/sites-enabled/*
Works ! This lists the targets of the symlinks.
Quick'n'dirty tip : this seems to work better if you put a * at the end of the symlink (either regular file or directory) you'd like de-referenced.
-I pattern --ignore=pattern Do not list entries matching the shell pattern pattern
-i --inode show the inode number
-L --dereference when "ls-ing" a symbolic link, show information for the target of the link, not for the link itself
-l use a long display format. By default, this mode displays the modification time. Display the access time with -u.
-n --numeric-uid-gid show UID and GID in numerical form instead of showing names.
-R --recursive list subdirectories recursively
-r --reverse reverse order while sorting
-s --size print the allocated size of each file, in blocks
-S sort by DESC file Size
-t sort by modification time, newest first.
-u use time of last access instead of last modification while sorting (-t) or printing (-l) (source)
-U do not sort : list entries in directory order.

Example :

List files and sort them by DESC size :

  • ls -Shl
  • ll -Sh
ls is smart enough to sort files by size even though they are displayed in human-readable format.

List directories only :

ls -ld */

Show seconds of last modification :

  • ls -l --time-style='+%d-%m-%Y %H:%M:%S' file
  • or : ls -l --time-style='full-iso' file

Show seconds of last access :

  • ls -ul --time-style='+%d-%m-%Y %H:%M:%S' file
  • or : ls -ul --time-style='full-iso' file

ls with shell patterns : lists 2010* and 2011* files ( source 1, 2 )

ll 201[01]*

Sum up the size of files returned by ls :

ls -l 201[01]* | awk '{ TOTAL += $5} END { print TOTAL/1024/1024 " MB"}'

Hack :

How to hide the total line when using ls -l (source) ?

You could run :

  • ls -l | grep -v '^total'
  • ls -l | tail -n+2
but that would be inelegant

-l causes the display of the total line. To workaround this, add -d * to your ls command.

less

Flags :

Command-line options :

Flag Usage
+F Follow file contents in real time, like tail -f does : less +F myFile
-i make further searches case-insensitive provided the search pattern is all lowercase !
-N show line Numbers
-R interpret ANSI color escape sequences
When it's given the --color=auto directive (which is often set by default in ~/.bashrc), grep & al. are smart enough to refrain from sending characters that less is unable to process. To force colorized output, and see it in less, do :
grep --color=always searchPattern myFile | less -R
-S --chop-long-lines show the portion of a long line that doesn't fit the screen width on the next line
chop long lines in interactive mode

Interactive commands :

Flag Usage
F Follow file contents in real time, like tail -f does
g
ng
jump to beginning of line
jump to line n
G jump to end of line
- i toggle case sensitivity in searches (by default, searches are case-sensitive, so this will turn them case-insensitive at first strike)
n jump to next match
N jump to previous match
v edit the current file with the editor defined in the EDITOR environment variable
:-n display the next file (when less is launched like : less *conf)
:-p display the previous file (when less is launched like : less *conf)
&RegExp show lines matching RegExp. Back to normal (display all lines ) : &
/pattern search forward pattern (see also)
?pattern search backward pattern
  • =
  • : f
  • Ctrl-g
display information about the file being viewed :
  • line range currently on screen / total line number
  • byte offset / file size (aka byte count)
  • point in file as percentage

Example :

Chop long lines :

- - c h

View myFile starting at the line matching pattern :

less +/pattern myFile

lsof

Usage :

List open files.
Given that in Unix everything is a file, lsof can also be used to deal with processes and connections.

Flags :

Flag Usage
-a Logical and for all the filtering parameters, wherever the -a is placed in the command line.
-c command List files opened by processes fired by a command starting with command. This can be :
  • a single string : lsof -c python
  • an excluded string (marked with ^). But this is especially useful with -a : lsof -c gnome -a -c ^gnome-pan
  • a regular expression : lsof -c /RegEx/ (defaults to extended regular expressions)
+D /path/to/dir Search for all open instances of directory /path/to/dir (i.e. users having cded into it) and all the open files and directories it contains to its complete depth (= list recursively open files under /path/to/dir)
-i address List open files related to internet services and matching address. If address is not specified, then * is assumed.
address can be specified as : [4|6][protocol][@hostname|hostaddr][:service|port]
+Ln
-Ln
  • enable (+) or disable (-) ...
  • the listing of file links count.
  • +Ln : list files having <n links. +L1 will select open files that have been unlinked (i.e. deleted)
  • example
-n Inhibits the conversion of network numbers to host names for network files. Inhibiting conversion may make lsof run faster.
-P Inhibits the conversion of Port numbers to port names for network files. Inhibiting conversion may make lsof run faster.
-p PID List files open by the process(es) matching PID. This can be :
  • a single number : 123
  • a list of numbers : 123,456 (no space allowed)
  • a list with exclusions (marked with ^) : 123,456,^789
-u foo Select users with login or UID matching foo. This can be a single, a list or a list with exclusions.

Example :

Basic examples

lsof -i tcp
list all processes related to TCP daemons
lsof -i udp
list all processes related to UDP daemons
lsof -i tcp:80
list all processes related to a web server
lsof -i @1.2.3.4
list network connections between the current host and the host with IP 1.2.3.4
lsof -i -a -p 1234
list internet connections opened by the process with PID 1234
lsof -u kevin,500 -p 123,456
list files opened by kevin or by whoever has the 500 UID or by PID 123 or by PID 456

List deleted files that have not yet been erased (details) :

  • lsof -nP | grep deleted
  • or : lsof +L1
COMMAND		PID	TID	USER		FD	TYPE	DEVICE	SIZE/OFF	NODE		NAME
...
iceweasel	5435	23628	stuart	50r	REG	254,1	887976		30938420	/home/stuart/.local/share/gvfs-metadata/home (deleted)
mozStorag	5435	5554	stuart	50r	REG	254,1	887976		30938420	/home/stuart/.local/share/gvfs-metadata/home (deleted)
localStor	5435	5557	stuart	50r	REG	254,1	887976		30938420	/home/stuart/.local/share/gvfs-metadata/home (deleted)
...
These files have been marked as deleted but the storage space they use has not been freed yet because they are still used by one or more processes. To free the space used by any of these files, you just have to stop/restart the process holding it.
For details on any of these "almost deleted" files, have a look at /proc/PID/fd/FD (FD : digits only, no trailing letter). To recover one of them, just read below.

"Deleted" files can be the reason why a volume is reported as full whereas ls, df and du report there is some space left.

This can be used to recover "almost deleted" files :
  1. In a terminal, run : echo "Hello World" > /tmp/myFile; less /tmp/myFile
    Using less in this example is the key, since it will "hold" the file as long as you don't quit less
  2. In an other terminal, run : rm /tmp/myFile; ls /tmp/myFile; lsof | grep /tmp/myFile, which will output :
    ls: Cannot access /tmp/myFile: No such file or directory
    less	3803		stuart	4r	REG		254,0	12	133754 /tmp/myFile (deleted)
    
  3. But this is not over yet. Try this : cat /proc/3803/fd/4. It will output :
    Hello World

    Please note that even though the reported file descriptor was 4r (like removed), the actual file name is 4.

  4. cp /proc/3803/fd/4 /saved/from/limbo : you just recovered a deleted file
  5. And when you quit less, ls /proc/3803/fd/4 says : ls: Cannot access /proc/3803/fd/4: No such file or directory

lsattr

Usage :

list file attributes

Flags :

Flag Usage
-d Display attributes of a directory itself, not those of its contents.

File attributes are a combination of :

Option Usage Effect
A no atime update the file "access time" is not updated when accessing the file
a append only the file can be only opened in "append for writing" mode
c compressed the file is automatically compressed on the hard drive by the kernel. data is uncompressed/compressed on the fly to read/write the file
D synchronous directory updates modifications on a directory having this attribute are synchronously written to the hard drive.
d no dump the file will NOT be backuped by the dump utility
E compression error experimental compression patches use this attribute to indicate the file has a compression error. This attribute can not be added/removed by chattr
e uses extents the file is using extents for mapping the blocks on disk. It may not be removed using chattr
I index into hash trees Applies to directories only. This attribute can not be added/removed by chattr
i immutable the file can not be changed, written, deleted, renamed. It can not be targeted by a link.
Applying this flag requires root privileges.
j data journalling data is written to the Ext3 journal before being written to the file
S synchronous updates when modifying the file, modifications are written synchronously to the disk
s secure deletion when deleting the file, its blocks are filled with zero's and written to the disk
T top of directory hierarchy States this directory is the filesystem root
t no tail-merging the file has no partial block at the end merged with others file's ends.
u undeletable upon deletion, the contents of the file is backuped so that it can be restored
X direct access to compressed file brute content used by experimental compression patches. This attribute can not be added/removed by chattr
Z "dirty" compressed file used by experimental compression patches. This attribute can not be added/removed by chattr

Example :

get attributes of a standard file :

lsattr myFile
------------- myFile

get attributes of a directory :

lsattr -d myDirectory
------------- myDirectory