Awk commands - Description, flags and examples

mail

strtonum

strtonum(string) returns the numeric value of string
mail

int

int(x) returns the nearest integer to x, located between x and zero and truncated toward zero. Examples :
mail

continue

mail

length 

awk ' \
	
	{ if(length(dataField) > 0) print dataField } \
	 \
	' file.log
mail

gsub

Usage

gsub(regexp, replacement [, target])

Example

replace all o with 0 :
echo 'hello world' | awk '{ gsub(/o/, "0"); print }'
hell0 w0rld
replace all e with E in the 2nd word only :
echo 'happy halloween' | awk '{ gsub(/e/, "E", $2); print }'
happy hallowEEn
remove square brackets [] :
echo '[foo]' | awk '{ gsub(/[\[\]]/, ""); print }'
foo
mail

gensub

Usage

Like gsub, gensub allows regex-based search and replace. Some of its notable differences are :
gensub(regexp, "replacement", how [, target])

Example

Basic examples :

  • echo 'hello world' | awk '{ result=gensub(/o/, "O", 1); print result; }'
    hellO world
  • echo 'hello world' | awk '{ result=gensub(/o/, "O", "g"); print result; }'
    hellO wOrld
    The variable name passed to print need not (dictionary.cambridge.org, english.stackexchange.com) a leading $.
  • to extract a substring, the regex must wrap the whole line: 
    testString='Lorem ipsum dolor sit amet, consectetur adipiscing elit';
echo "$testString" | awk '{ result=gensub(   /(ipsum)/,    "\\1", "g"); print result }'
echo "$testString" | awk '{ result=gensub( /.*(ipsum).*/,  "\\1", "g"); print result }'
echo "$testString" | awk '{ result=gensub(/^.*(ipsum).*$/, "\\1", "g"); print result }'
    Lorem ipsum dolor sit amet, consectetur adipiscing elit
ipsum
ipsum

A not-so-basic example :

On lines starting with a vowel, change the 3rd character of the 2nd word into an X :
echo -e 'alpha bravo\ncharlie delta\necho foxtrot' | awk '/^[aeiouy]/ { result=gensub(/^(..).(.*)/, "\\1X\\2", 1, $2); print $1" "result; next; } { print }'
alpha brXvo
charlie delta
echo foXtrot
mail

print

mail

print, printf, sprintf

print :

  • with no argument, print the whole input line :
    echo -e 'line 1\nline 2\nline 3' | awk '{print}'
    line 1
line 2
line 3
  • with 1 argument, print it :
    echo -e 'line 1\nline 2\nline 3' | awk '{print $2}'
    1
2
3
  • with more than 1 argument :
    • when the arguments are separated by commas : print arguments separated by SPACE (default) or the specified OFS
    • when the arguments are separated by spaces : print arguments concatenated
    echo -e 'line 1\nline 2\nline 3' | awk '{print $2,$1}'; echo -e 'line 1\nline 2\nline 3' | awk '{print $2 $1}'
    1 line
2 line
3 line
1line
2line
3line
  • Appends a carriage return \n to the output (printf doesn't) :
    echo | awk '{print "Hello world"}'; echo | awk '{printf "Hello world"}'

printf :

  • Doesn't add a trailing \n
  • Supports the C-style printf(string, expression list) syntax :
    echo | awk '{printf("%d is The Answer to The Great Question.", 42)}'

sprintf :

Returns without printing, what printf would have printed out with the same arguments.
mail

switch case

Here's a very basic example (not-so-perfect but you'll get the idea ) : 
echo 'abc' | awk '{
	switch ($0) {
		case /[[:lower:]]+/:
			print "lowercase"
			break
		case /[[:upper:]]+/:
			print "uppercase"
			break
		}
	}'
As a one-liner that can be pasted into the shell : 
echo 'abc' | awk '{ switch ($0) { case /[[:lower:]]+/: print "lowercase"; break; case /[[:upper:]]+/: print "uppercase"; break; } }'
mail

next

immediately stop processing the current record and go on to the next one
mail

exit

see examples : 1, 2
mail

break

mail

if else 

checkPageExists() {
	local page=$1
	curl -sI "$page" | awk '/^HTTP\/1.1/ {
		if ($2=="200")
			exit 0
		else
			exit 1
		 }'
	}

main() {
	
	for page in $pageList; do
		checkPageExists "$page" || continue
		
	done
	}
mail

system

A basic example

echo -e 'line A\nline B\nline C\nD' | awk '/^line/ { system("echo "$NF) }'
  • this command is absolutely useless : I just needed a dummy working example
  • don't forget that Awk variables must stay outside of quotes

How to store the result of a system command into a variable ? (source)

There are several methods to run a shell command from Awk :

with system(myCommand)

myVariable=system(myCommand) stores the return code of myCommand into myVariable
  • awk 'BEGIN { result1=system("true"); result2=system("false"); print result1" "result2; }'
    0 1

with myCommand | getline

awk 'BEGIN {
	myCommand = "date --iso-8601=seconds"
	myCommand | getline myResultVariable
	close(myCommand)
	print "Current date = "myResultVariable
	}'

Mixing Awk and Bash tests :

[ -e someFile ] && rm someFile; for i in 1 2; do
	echo -e 'hello someFile world' | awk '{ print "\ninput : "$0; if(system("[ -f "$2" ]")) { print $2" exists" } }'
	ls someFile; touch someFile
done; rm someFile
  1. this begins by making sure that the file someFile does not exist
  2. then there's a for loop that runs twice
  3. each run echoes some text to awk via a pipe :
    1. display the input value as-is
    2. make a shell-based test using system
    3. display some text accordingly
  4. ls to confirm someFile exists or not, then it is touched
  5. loop
  6. remove someFile at the very end (Boy scout rule)
input : hello someFile world
someFile exists							mmmkay
ls: cannot access 'someFile': No such file or directory		make up your mind !

input : hello someFile world
someFile
This is because Awk and Bash disagree on what makes a "success" :
success failure
Awk anything else 0
Bash 0 anything else
  1. when the file exists, [ -f "$2" ] is a Bash success : 0
  2. system returns this code as-is
  3. but for Awk : if(system(Bash success)) is false 
    echo | awk '{ if(system("true")) print "ok" }'
(nothing)

echo | awk '{ if(system("false")) print "ok" }'
ok
To make this work, the solution is to negate the result of the Bash test in the original command :
if(system("[ ! -f "$2" ]")) {