Bash Index : W - The 'W' Bash commands : description, flags and examples

1. Create a shell script like :

   #!/usr/bin/env bash
   # Simple chat system for local users
   nick=${1?Usage: $0 nickname (e.g. $0 bob)}
   log="/tmp/$0.log"
   >"$log"
   chmod a+w "$log"
   echo "^D to exit chat." >&2
   tail -F "$log" & tailPid=$!
   trap 'kill "$tailPid"' 0
   while IFS='' read -er line; do echo "<$nick> $line"; done >> "$log"

2. Users may join with : ./chat.sh bob
3. Don't forget to purge the $log file at the end

• whois google.fr
• whois pur.fr | less

• Use the quick-n-dirty "background process" method : wget url1 & wget url2 & wget url3
• consider tools like aria2 (source)
• hack it with xargs : cat "$listOfUrlsToDownload" | xargs -n 1 -P $numberOfParallelDownloads wget -q (source)
• hack it with parallel : parallel -j $numberOfParallelDownloads "echo -n .; wget -c -q {}" < "$listOfUrlsToDownload"

• ~/.wgetrc for user-specific settings
• /etc/wgetrc for system-wide settings

http_proxy = http://proxyUser:proxyPassword@proxyHostname:proxyPort/
	use_proxy = on
	wait = 15

Alternate shell-based solution (details) :

If proxyPassword contains some special characters such as [SPACE], : or @, you'll have to url-encode them.

wget --header="Authorization: Basic ZGJlcnJlYmlAcHJpc21hbWVkaWEuY29tOnByaXNtYTIwMTE=" ...

• Authentication should be done so that only one request is necessary. If something's wrong, you may get a HTTP 401, then a HTTP 200. Consider using the -S flag while developping.
• The value after the Basic, is the base64 encoding of the string login:password (source)

wget --header="Authorization: Basic "$(echo login:password | base64) ...

wget -q -S --header="Host: my.domainName.xxx" origin-my.domainName.xxx

This may not work if sent through a proxy, as this proxy may block HTTP requests having custom headers, or return the result of a previously cached request made without the custom header.

• with internal links
• without leaving the initial domain
• without climbing into parent directories

wget -k -r -E -np -p --restrict-file-names=ascii URL_of_start_page

WARNING: The certificate of 'www.example.com' is not trusted.
WARNING: The certificate of 'www.example.com' hasn't got a known issuer.

• install the root CA's certificates : apt-get install ca-certificates (source)
• or update them if they're already installed (as root) : update-ca-certificates (source)

which wget
/usr/bin/wget

which bc
/usr/bin/bc

echo						just to split pasted commands from their output and make this example more readable
alias echo='echo "ECHO SAYS:"'	defines an alias for echo named exactly echo, hence the need to discriminate echo (the alias) from echo (/usr/bin/echo)
echo 'hello world (with the alias)'
$(which echo) 'hello world (regular echo from which)'
unalias echo
echo 'hello world (unaliased)'

ECHO SAYS: hello world (with the alias)
hello world (regular echo from which)
hello world (unaliased)