jq - Process JSON from the CLI

mail

How to pass a variable to jq ?

To do so :
jq --arg variableName variableValue 'jq expression with $variableName'
searchedFirstName='Peter'
cat << EOJSON | jq -r \
	--arg needle "$searchedFirstName" \
	'.users[] | select(.first==$needle) | .last'
{
	"users": [
		{ "first": "Bruce", "last": "Banner" },
		{ "first": "Peter", "last": "Parker" }
	]
}
EOJSON
Parker
mail

How to parse and display multiple fields ?

Table of contents

values only

cat << EOJSON | jq -r '.users[] | "\(.first) \(.last)"'
{
	"users": [
		{
			"first": "Bruce",
			"last": "Banner"
		},
		{
			"first": "Peter",
			"last": "Parker"
		}
	]
}
EOJSON
gives : 
Bruce Banner
Peter Parker
alternate syntax : 
 | jq -r '.users[] | .first + " " + .last' {

to produce a CSV

It's still possible to use the syntax above and insert commas , : 
 | jq -r '.users[] | "\(.first),\(.last)"' {
But jq can do it for you : 
cat << EOJSON | jq -r '.users[] | [ .first, .last ] | @csv'
{
	"users": [
		{
			"first": "Bruce",
			"last": "Banner"
		},
		{
			"first": "Peter",
			"last": "Parker"
		}
	]
}
EOJSON
gives : 
"Bruce","Banner"
"Peter","Parker"

column prefixes and values

cat << EOJSON | jq -r \
	--arg firstName 'First Name : ' \
	--arg lastName 'Last Name : ' \
	'.users[] | $firstName + .first + ", " + $lastName + .last'
{
	"users": [
		{
			"first": "Bruce",
			"last": "Banner"
		},
		{
			"first": "Peter",
			"last": "Parker"
		}
	]
}
EOJSON
gives : 
First Name : Bruce, Last Name : Banner
First Name : Peter, Last Name : Parker

display a "common" value, then lower values

cat << EOJSON | jq -r '.users[] | [ .id, .data.first, .data.last ]'
{
"users": [
	{
	"id" :   123,
	"data" : { "first": "Peter", "last": "Parker" }
	},
	{
	"id" :   345,
	"data" : { "first": "Bruce", "last": "Banner" }
	}
	]
}
EOJSON
[
  123,
  "Peter",
  "Parker"
]
[
  345,
  "Bruce",
  "Banner"
]
with an extra | jq -r '.users[] | [ .id, .data.first, .data.last ] | @csv' { : 
123,"Peter","Parker"
345,"Bruce","Banner"
mail

jq

Usage

jq is like sed for JSON data — you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

Flags

Flag Usage
--arg name value pass a value to jq as a predefined variable (examples : 1, 2)
-c --compact-output by default, jq pretty-prints JSON output. Using this option will result in more compact output by instead putting each JSON object on a single line.
-M --monochrome-output disable color : by default, jq outputs colored JSON if writing to a terminal
-r --raw-output output string without quotes

Example

List Firefox extensions :

  • firefoxExtensionsFile=$(find ~/.mozilla -name "*extensions.json"); jq -r '.addons[].defaultLocale.name' "$firefoxExtensionsFile"
    extension 1
extension 2
extension 3
  • count extensions :
    firefoxExtensionsFile=$(find ~/.mozilla -name "*extensions.json"); jq -r '.addons | length' "$firefoxExtensionsFile"
    16
  • add some text to the output (source) :
    firefoxExtensionsFile=$(find ~/.mozilla -name "*extensions.json"); jq -r --arg name 'Extension Name : ' '$name + .addons[].defaultLocale.name' "$firefoxExtensionsFile"
    Extension Name : extension 1
Extension Name : extension 2
Extension Name : extension 3

resultFile='./result.json'; taigaApiUrl='https://taiga.example.com/api/v1'; taigaProjectId='2949'; username='bob'; password='123456789'

authToken=$(curl -s --noproxy "*" --insecure -X POST -H "Content-Type: application/json" -d '{ "username": "'$username'", "password": "'$password'", "type": "normal" }' "$taigaApiUrl/auth" | jq -r '.auth_token')

[ -f "$resultFile" ] && rm "$resultFile"
curl -s --noproxy "*" --insecure -X GET -H "Content-Type: application/json" -H "Authorization: Bearer ${authToken}" "$taigaApiUrl/userstories?project=$taigaProjectId" > "$resultFile"

# List the user stories IDs :
# jq '.[29].ref' result.json
# for i in {0..29}; do jq ".[$i].ref" result.json; done

# [] is an iterator, no need of 'for' loop \o/
# https://stedolan.github.io/jq/manual/#Basicfilters
#jq '.[].ref' result.json
#jq ".[].subject" result.json

# to get both :
# source : https://stackoverflow.com/questions/28164849/using-jq-to-parse-and-display-multiple-fields-in-a-json-serially#answer-31418194
#jq -r '.[] | "\(.ref) ==> \(.subject)"' result.json

# get tags :
# thanks to : https://stackoverflow.com/questions/41712363/basic-jq-usage-how-to-get-nested-value
#jq -r '.[].tags | .[] | .[0]' result.json

echo 'Tags :'
jq -r '.[].tags | .[] | .[]' "$resultFile" | sed '/^null$/d' | sort | uniq -c | sort -nr